How does batch size affect model size? I'm suffering from a significant brain fart while trying to get my head around how does batch size affect overall model size e.g for CNNs. Does it serve as an additional dimension for all the weight tensors?

Considering:

VGG16 model
batch_size of 16
image size of 224x224x3
conv_1 being the initial 1x1 convolution with stride 1 and 3:64 channels mapping

The input will be a tensor of [16, 224, 224, 3] shape. Will the output of convolution layer be [16, 224, 224, 64] and therefore - will all the weights have additional 'batch size' dimension and thus - impose a linear increase of model size with respect to the batch size?

The model size (i.e., the number of parameters) is completely independent of the batch size you use during the training.

Whatever the batch size is, your model processes each input within the batch independently. Concretely, batch size affects how many samples your model has to process before it makes an update.

For example, when batch size is 1, it computes the gradients and updates the parameters for each input. When batch size is the whole dataset, it computes the gradients for each input in the dataset independently. It then takes the average of those gradients and uses this average to update the model, so there will be a single update per epoch.

In your case, the model takes an input of size (width=224, height=224, color_channels=3) whatever batch size you use during the training.


Can AI help summarize article or abstract sentence keyword? I'm wondering if AI now can help us abstract summary or general idea of long article, for example novel or historical stories, or abstract most important keyword from sentence;

Would you please tell me if any of this kind of project is done?

I wish I can improve my reading speed and effectiveness with AI help.

Yes. Text summarisation has been a research topic in (computational) linguistics for literally decades. Have a look at the Wikipedia page on Automatic Summarisation for an overview.

There are basically various different approaches: either, selecting salient sentences (or parts of sentences) which represent the gist of the text, or, trying to 'understand' the text and generating new sentences. The former is generally easier, and works on any text, while the latter would probably be able to produce better results, but is more complicated and would not work on any text, as it would be specific to a particular topic.


Why do some DQN implementations not require random exploration but instead emulate all actions? I've found online some DQN algorithms that (in a problem with a continuous state space and few actions, let's say 2 or 3), at each time step, compute and store (in the memory used for updating) all the possible actions (so all the possible rewards). For example, on page 5 of the paper Deep Q-trading, they say

This means that we don't need a random exploration to sample an action as in many reinforcement learning tasks; instead we can emulate all the three actions to update the Q-network.

How can this be compatible with the exploration-exploitation dilemma, which states that you have to balance the time steps of exploring with the ones of exploiting?

The example that you linked is using a model (emulation) in order to look ahead at all possible actions from any state. It essentially explores off-policy and offline using that model. This is not an option that is available in all environments, but if possible it resolves the exploration/eploitation dilemma for a single time step nicely by investigating all options.
Longer term the agent proposed by the link does not sufficiently explore for general use in my opinion. It appears to always choose a single action deterministically based on maximising action value. In other words it always attempts to exploit the training data so far, even though it augments the training data with short-term knowledge about exploration. However, this appears to be sufficient in the problem domain that it is used in. I suspect this is for a couple of reasons:

The environment is non-stationary, making long-term state predictions unreliable in any case. An agent that learns to exploit in the short term (i.e. over only a few time steps into the future) is likely to be approximately optimal already.

State transitions may be highly stochastic, meaning that state space will still be adequately explored even using a deterministic policy. This feature of the environment is also used by other well-known Q learning approaches with deterministic behaviour policies, such as TD Gammon

I think you have correctly identified a weakness of the approach used in the linked paper that means it may not make a strong general algorithm. The algorithm avoids addressing the exploration/exploitation balance in full, and instead relies on features of the environment to work well despite this. If you find yourself working in similar environments for your own projects, then it may well be worth trying the same approach. However, if you find yourself working in a more deterministic environment with more stationary behaviour and sparse rewards, the lack of state space exploration would be a serious limitation.


What kind of problems cannot be solved using machine learning techniques? For the problems that can be solved algorithmically.

We have very good formal literature for which problems can be solved in polynomial, exponential time and which cannot. P/NP/NP-hard

But do we know some problems in machine learning paradigm for which no model can be trained? (With/without infinite computation capacity)

At least you should be aware of two points:

P/NP/NP-hard (and all other class of complexities) are thoroughly valid for the machine learning area as well. Because these concepts are related to the fundamental of computations (theory of computation), and machine learning is not an exception here.
One of the useful concepts in the complexity of the learning problem is the VC dimension, PAC learnability, and their related concepts (such as sample complexity). Although these concepts can't be enough to measure the time complexity, they are useful for finding the learner model's capacity.


Get object's orientation or angle after object detection I'm trying to get a detected car's orientation when object detection is applied. For instance, when we apply object detection on a car and get a bounding box, is there any ways or methods to calculate where the heading is or the orientation or direction of the car (just 2D plane is fine)?
Any thoughts or ideas would be helpful.

I think the problem can be phrased (more generally) as a Pose Estimation Problem.  That term might help in obtaining better search results when searching for relevant papers.
One paper that I found on the given topic was this one. Even if it is maybe (for whatever reason) not what you are looking for precisely, it might still contain valuable references to different techniques that might be applicable to your problem at hand.


How to design a fitness function for a problem where there are 2 objectives? I am told to express a fitness function for a question I have been presented. I am unsure how I would express the function. In words, what I have written down makes sense but turning this into a mathematical formula is proving a bit difficult. My understanding is:
The fitness function for this scenario will want to ensure that the best offer for building the computers is chosen whilst the price of the final optimal offer is low.
The fitness function in this case would want to consider a few factors. The first factor is that the quantity of the computer parts was enough that each offers that were returned had a sufficient quantity of parts. Ideally, it would be best if we did not have any duplicates of parts in the offers. The cost is low too, but all parts have been found amongst the different offers that we have.
The fitness function will need to ensure all of this is factored in.
The scenario and question are below:

For the production of a number of laptops, a computer company needs a quantity of each component such as screens  (S),  hard drives  (HD),  optical drives  (OD),  RAM,  video cards  (VC),  CPU,  Ports,  etc.  The company received a number of priced offers. Note that offers do not contain all components. As examples:

Offer 1: 1000 RAMs, 800 HDs, 2000 ODs – £75K
Offer 2: 1850 S, 1570 OD - £40K
Offer 3: 3000 HD, 2000 RAM – £70K
Offer 4: 1500 RAM, 2000 VC, 1700 S – £55Ketc.

The company would be interested to accept cheaper offers in the first place. Answer the following: Give the expression of the fitness function.

Any help would be greatly appreciated .

If we assume that each laptop requires 1 component of each type, so 1 screen, 1 hard drive, 1 RAM, etc. (i.e. the company has no preference for the type of component), then the company, to maximize the number of laptops it can build (which is supposedly the ultimate goal of the company), it should

maximize the number of instances of the least available component in the offer, and

minimize the cost of the offer

So, one possible fitness function would then need to take these 2 objectives into account, so this would be a multi-objective optimization problem.
Recall that a fitness function evaluates individuals in the population. If we assume that the individuals are the offers and the goal would be to find the best offer in the space of offers, then we can devise some fitness function $f$ of the form
$$
f(o) = (1 - \alpha)\frac{1}{1 + p(o)} + \alpha \min(o), \label{1}\tag{1}
$$
where

$o$ is an offer, so $f(o)$ is the fitness of the offer $o$, i.e. how much it is desirable (i.e. we want the fitness to be high, usually).
$p(o)$ is the price/cost of the offer $o$
if we assume that $o$ is an array of the form $o = [1000, 800, 2000, \dots]$, where $o_i$ is the number of items of type $i$ (e.g. $i$ is RAM, then $o_i$ would be the number of RAMs in the offer $o$), then $\min(o)$ would be the number associated with the least available item in the offer (e.g. if the indidivudal was $[12, 4, 5]$, then $\min(o) = 4$)
$\alpha \in [0, 1]$ is a hyper-parameter that trades off the two objectives (i.e. prices and the smallest number)

The fitness function $f$ in equation \ref{1} will be maximal when $p(o) = 0$ and $\min(o) = N$, where $N$ is some maximum threshold of possible number of items of the same type that an offer can have.
I have just come up with this fitness function. I don't know whether it will work in practice or not, but this is the idea of what you have to do.  You can design other similar fitness functions (for example, how would you design a fitness function that takes into account that the number of components of each type should be more or less the same?). You may want to try to implement this e.g. with DEAP and see how it behaves. You probably also want to make sure that $o$ are arrays of integers (and not floating-point numbers). You probably also want to read this answer.


Reinforce Learning: Do I have to ignore hyper parameter(?) after training done in Q-learning? Learner might be in training stage, where it update Q-table for bunch of epoch.

In this stage, Q-table would be updated with gamma(discount rate), learning rate(alpha), and action would be chosen by random action rate.

After some epoch, when reward is getting stable, let me call this "training is done". Then do I have to ignore these parameters(gamma, learning rate, etc) after that?

I mean, in training stage, I got an action from Q-table like this:

if rand_float < rar:
    action = rand.randint(0, num_actions - 1)
else:
    action = np.argmax(Q[s_prime_as_index])

But after training stage, Do I have to remove rar, which means I have to get an action from Q-table like this?

action = np.argmax(self.Q[s_prime])

Epsilon (your rar parameter) is used to have the possibility of a random action which handles the problem of exploration and exploitation.

If you have a deterministic world and are certain that your learner has learned a policy with performance to your liking, removing it will ensure that only the greedy policy being learned will be taken and that you can expect the same performance afterwards.

The agent will still be learning at this point because alpha (learning rate) is not set to 0. Because of this, if the world is stochastic instead and possibly changes slowly over time (e.g. motor speed due to battery power), then having epsilon non zero still allows the agent to explore and overcome these changes when they happen before it's performance gets low enough to change is behaviour (e.g. q values dropping).

If instead you simply want to test the performance of the policy learned up until a timestep, setting alpha to 0 in a compares the epsilon-greedy policy, setting both alpha and epsilon to 0 compares the greedy policy.


Can a neural network learn to predict a number given a binarized image of a rectangle? Let's assume that we have a regression problem. Our input is just binarized image that contains a single rectangle and we want to predict just a float number.  Actually, this floating-point number depends on rectangle angle, rectangle size and rectangle location. Is this problem can be solved by a neural network?

I think, it can not be solved by a neural network, because rectangle angle, size and location are latent variables and without learning these latent variable, above problem can not be solved. What do you think?

It can definitely be learned, the question is the approach. It would be expensive and difficult from a modeling directive to do this Densely, so usually convolutions are the way to go. An issue with convolutions is that is generally focuses on equivariant and relative features, so if you need specific location within the approach might be worth the simple alteration of CoordConv. Regardless of approach, that type of input to output is possible, you just have to consider it when modeling.


Evolving Machine Learning It seems to me that, right now, the key to making a good Machine Learning model is in choosing the right combination of hyper-parameters.

Firstly: Am I right in saying, if a model is able to tune it's own hyper-parameters, we have in some sense achieved a general intelligence system? Or a glimpse of an actual artificial intelligence system?

I feel the answer lies in what one means by " tune it's own hyper-parameters". If it means to be able to reach Bayesian levels of performance on that task then theoretically, after the tuning, the model is able to perform at par or better than humans and so it seems the answer would be yes.

Secondly: I understand that hyper-parameter tuning is done intuitively. But there are a  set of general directions that is discernable looking at results.  Here I am talking about a heuristic approach to perfect a learning model.
Consider an example:
Say I hardcode a model to, while training, observe gradient values. If the gradient is too large or the cost is highly oscillatory, then restart training with a smaller learning rate.
Then obtain metrics on a test set. If it is poor, then again restart training with regularisation or increased regularisation.
It can also observe various plot behaviours, etc.

The point is maybe not every trick up a researcher's sleeve can be hardcoded. But a decent level of basic tuning can be done.

Thirdly: Let us say, we have a reinforcement learning system on top of a supervised learning system. That is an RL network sets some hyper-parameters. The action then is to train with these hyper-parameters. The reward would be the accuracy on the test set.

Is it possible that such a system could solve the problem of hyper-parameter tuning?

Such a system can and does solve the problem of hyperparameter tuning. Google's AutoML does this. Here is another example that uses a Genetic Algorithm to breed new neural network structures.
AutoML has been shown to outperform humans in the rate that it improves network designs. It seems to favour Residual Network style topologies.


What does a CNN classifier default to? I've been messing around with an Open Set, Binary Classifier and am having trouble with it.  I'm sure there are a lot of reasons for that trouble.

One thing I am struggling with is, what does the model predict if it has never seen the image before? An example would be if I'm trying to detect sheep across all background scenes.  If I train a binary classification set with one class having lots of sheep in it and the other class having lots of various backgrounds, what would the model predict if it came across a background it had never seen before with no sheep in it?  [mine is telling me "sheep" and I don't know why]

Thanks, Mecho

I am assuming the images you gave the model all contain sheep. This is what i understand from your question.

Any model that you build will be based on the data that you give (training data) and your code. In your case, if you only give images that contains sheeps, and then you test it with no sheep and a background the model hasn't seen, it will search through all of its node, derived from your training data, to see which one is closest to the image you gave. Based on the information given, the only 'route' your cnn model can take is the one with sheeps on it, because you only gave images of sheeps for it to learn.

Here are a few suggestions that i can give you:

Give your model images with no sheep and different backgrounds so it can handle cases where there are no sheeps
Or you can add a piece of code to your model that tells it to default to some value if certain conditions aren't met (If the model doesn't 'see enough' sheep and background, it defaults to 'unknown image' or whatever you choose)

You are on the right path though! Since you are playing around with a binary classifier, you should definitely feed images with and without sheeps so that it can identify the two cases. Remember, a binary classifier is optimal when you give it 2 things to look out for, such as identifying images with and without sheep.

Here are some reading materials that you can brush up on to get a better basic understanding of how CNN works, personally i found the video helpful on the second link:

https://becominghuman.ai/what-exactly-does-cnn-see-4d436d8e6e52
https://brohrer.github.io/how_convolutional_neural_networks_work.html

Also, i find google images helps me alot in terms of visualizing binary classifier as a concept.


What is the best way to smoothen out a loss curve plot I am currently using a loss averaged over the last 100 iterations, but this leads to artifacts like the loss going down even when the current iteration has an average loss, because the loss 100 iterations ago was a large outlier.

I thought about using different interval lengths, but I wonder if an average over the last few iterations really is the right way to plot the loss.

Are there common alternatives? Maybe using decaying weights in the average? What are the best-practices for visualizing the loss?

You can use the Exponential Moving Average method. This method is used in tensorbaord as a way to smoothen a loss curve plot.

The algorithm is as follow:

However there is a small problem doing it this way. As you can see S_t is initialized with the starting value, which makes the starting curve inaccurate.

The green curve is the ideal curve for the algorithm, but the purple curve is the predicted curve. The curve is not correct on the start. To solve this, a correction factor is added in, thus making the algorithm this:

This introduces WeightedCount which decreases over time to 0.

Exponential Moving Average is also used is other areas of deep learning, the most notable being some optimization algorithms. It is used in Adam, RMSProp and other similar optimizers to smooth out the gradients to make the path to minimal loss a more direct and straightforward path.


Inner working of Bidirectional RNNs I'm trying to understand how Bidirectional RNNs work.

Specifically, I want to know whether a single cell is used with  different states, or two different cells are used, each having independent parameters.

In pythonic pseudocode,

Implementation 1:

cell = rev_cell = RNNCell()
cell_state = cell.get_initial_state()
rev_cell_state = rev_cell.get_initial_state()
for i in range(len(series)):
    output, cell_state = cell(series[i], cell_state)
    rev_output, rev_cell_state = rev_cell(series[-i-1], rev_cell_state)
    final_output = concatenate([output, rev_output])

Implementation 2:

cell = RNNCell()
rev_cell = RNNCell()
cell_state = cell.get_initial_state()
rev_cell_state = rev_cell.get_initial_state()
for i in range(len(series)):
    output, cell_state = cell(series[i], cell_state)
    rev_output, rev_cell_state = rev_cell(series[-i-1], rev_cell_state)
    final_output = concatenate([output, rev_output])

Which of the above implementations is correct? Or is the working of Bidirectional RNNs completely different altogether?

The second implementation looks more correct and inline with how Bidirectional is defined. Specifically, bidirectionality doen't change the forward/backward logic of either direction, and just merges (concat/sum/...) the outputs of forward/backward at a matching timestep t.

You can check how Keras implements it here. There are distinct self.forward_layer and self.backward_layer that are initialized separately.

Your for loop doesn't look ok though. To calculate the output at time step 0 you'd have to calculate forward(0) and backward(n), which means you have to run backward on all the samples first. In practice, each direction is calculated separately and the results are merged afterwards. Check the implementaion in Keras here.


Do deeper residual networks perform better or worse? If you have an $18$ layer residual network versus and a $32$ layer residual network, why would the former do better at object detection than the latter, if you have both models are training using the same training data?

Just by having more parameters, the deeper model has a higher capacity than the smaller one. This means that theoretically it can learn to extract more complex features from the data. Additionally, more layers means that the model can extract even higher-level features from the data.
So, generally speaking, deeper models will most of the times outperform shallow ones for more difficult tasks.

The downside is that if you have a small amount of data, a high-capacity model has the ability of memorizing the training set, which would lead to overfitting.
Besides performance, deeper models require better hardware and larger training times. So, there are plenty of reasons for one to prefer a more shallow model to a deeper one.


How to know whether the object is moving after it is being detected? If my algorithm detects the type of object, how should I know if that object is moving or not? Suppose a person carrying an umbrella. How to know that the umbrella is moving?

I am working on a project where I want to know whether that particular object belongs to the person entering inside the store.
I was thinking about the bounding boxes(bb) approach where if the person's bb overlaps with the object's bb. But the problem arises when there are multiple objects with a person.

I would appreciate your help. Thanks.

Either (a) re-recognize it in subsequent frames and then train a network employing a trajectory model to the changing object model parameters or (b) recognize the object and its motion in a single object-trajectory parameterized model from the sequence of sets of detected edges indicating edge movement in the sequence of frames.


Why didn't champion of the Go game manage to win the last game against AlphaGo, after winning the 4th one? In the documentary about the match, it is said that after losing the 4th game, AlphaGo came back stronger and started to play in a weird way (not human-like) and it was pretty impossible to be beaten. Why and how did that happen?

The technique used by AlphaGo is "Monte Carlo Tree Search", combined with a very well trained neural network. The network's job is to estimate the quality of different board states and moves. This estimation is deterministic. If you show AlphaGo the same board on two different occasions, it thinks it is exactly as good (or bad) on both occasions.

Monte Carlo Tree Search however, is a randomized algorithm. So as a simplified explanation, the way AlphaGo decides which moves to make is:

Look at the current board.
Pick a random move, and imagine what the board would look like if you made that move.
Pick a random move for your opponent, and imagine what the board would look like if they made that move.
Keep doing steps 2 & 3 for a while, so that we're imagining being at some board many moves in the future along a random line of play.
Ask the neural network how good this board state is.
Repeat steps 1-5 many times. Then make whichever move led to the best lines of play on average, to make right now.

What this means is, AlphaGo won't always play the same way, because it doesn't actually consider every move explicitly. It just thinks about enough lines of play to be pretty confident about whether one move is better than another. This is actually not so far removed from how humans play most of the middle of games like these.

So, in game 4, essentially, Sedol got lucky. The random lines of play that AlphaGo chose to look at did not capture some critical facts about one or more board states. This led it to make a mistake. If you asked it to play the same game through again, it might not make the same mistakes (it might think about different random lines of play, that do capture the critical facts it missed in the first game). Further, it might choose to play slightly differently on other moves, which could have a big impact on the rest of the game. These two factors prevented Sedol from simply playing game 4 over again.


Can a neural network be used to detect sine waves? I am recording the vibrations of an AC Motor (50Hz Europe) and I am trying to find out whether it is powered on or not. When I record these vibrations, I basically get the vibration values ($-1$ to $+1$) over time.

I would like to develop a program to detect the presence of a 50Hz sine wave on a steady stream of input data. I will have $X$ and $Y$ measurements, where $X$ represents amplitude and $Y$ the time (sampled at 100Hz - it is possible to increase the sample rate to 200Hz or 400Hz at max)

Is this a task suited for a neural network, and if so, would it be less efficient than other means of detection?

Is this a task suited for a neural network

Yes. You have choices in fact:

A fully-connected network would be simplest architecture, and would work if you gave it some time window of samples (e.g. every 0.5 seconds or every 50 samples) and supervised training data - sets of samples with sensor readings and the ground truth value of whether the motor was on or not.
A 1D convolutional neural network would likely be most efficient and robust to train, and would take the same inputs and outputs as the fully-connected network.
A recurrent neural network would be tricker to train, but a nicer design as you could feed it samples one at a time. The input would be the current sample, and output the probability that the motor was on. When training this, you would also want to provide it transitions between the motor being on and off. The nice feature about this is that it should give you quick feedback about whether the motor was on or off - with the caveat that it may be more likely to trigger intermittent false positives, so a little extra post-processing might be required.

All of the above require you to collect training data, ideally in situations identical to planned use of the detector. So if the motor is mounted somewhere that could experience other vibrations, a few of those kind of scenarios should be simulated with motor both on and off.

  and if so, would it be less efficient than other means of detection?

In terms of computing power and effort on your part, you may find that an off-the-shelf Fast Fourier Transform (FFT) library function with a simple threshold at your target frequency will make a robust and simple detector, with no need for a neural network.

Typically for specific frequency detection you would take a window of samples, adjust them (using e.g. Hamming window) to reduce edge effects which would appear as frequencies in the conversion, and then run FFT. This combination is so common that you may find it already combined in the FFT library. For more on this, you would want to ask in Signal Processing Stack Exchange, where use of FFT is well understood.

If the environment is noisy or the target frequency can drift (making it hard to set a simple threshold) then you could also combine FFT with a neural network. This combination can solve much more complicated signal detection, and is used in speech processing for instance.

  sampled at 100Hz - it is possible to increase the sample rate to 200Hz or 400Hz at max

For reliably detecting a 50Hz signal, I would say that 200Hz sample rate is minimum. The theorectical minimum is 100Hz (i.e. twice the signal frequency) but may give you problems with noise and the possibility that your sample points just happen to fall on low amplitude parts of the oscillations, making it look like the motor is off even when it is on.


How do we know if GPT-2 is a better language model? You may have heard of GPT2, a new language model. It has recently attracted attention from the general public as the foundation that published the paper, OpenAI, ironically refused to share the whole model fearing dangerous implications. Along the paper, they also published a manifesto to justify their choice: "Better Language Models and Their Implications". And soon a lot of media were publishing articles discussing the choice and its effectiveness to actually prevent bad implications. I am not here to discuss the ethical components of this choice but the actual performance of the model.
The model got my attention too and I downloaded the small model to play with. To be honest I am far from impressed by the results. Some times the first paragraph of the produced text appears to make sense, but nine times out of ten it is giberish by the first or the second sentence. Exemples given in the paper seems to be "Lucky" outputs, cherry picked by human hands. Overall, the paper may suffer from a very strong publication bias.
However, most article we can read on the internet seems to take its powerfulness for granted. The MIT technology review wrote:

The language model can write like a human

The Guardian wrote

When used to simply generate new text, GPT2 is capable of writing plausible passages that match what it is given in both style and subject. It rarely shows any of the quirks that mark out previous AI systems, such as forgetting what it is writing about midway through a paragraph, or mangling the syntax of long sentences.

The model appears generally qualified as a "breakthrough". These writings do not match my personal experimentation as produced texts are rarely consistent / syntactically correct.
My question is: without the release of the whole model for ethical reasons, how do we know if the model is really that powerful?

You have experimented with a small model (117M parameters). OpenAI has now released the medium sized model (345M parameters). Note that the full model has 1.5B parameters. Also note that GPT-2 has been trained on a tiny fraction of all available text. It's almost guaranteed that a larger model trained on more text will generate better text.

I have experimented with the 345M model, and I agree with The Guardian assessment. See for yourself:

Prompt:

  I want to generate beautiful music using machine learning. Do you
  think a computer will ever be able to compose something as novel as
  creations of Mozart, Bach, or Chopin?

Results (first four samples, not cherry picked):

  Yes, computers will probably be able to do it by 2020. I think the key
  for us will be to make music that is interesting and beautiful, so we
  will never really know. My hope would be that computers will become
  far more efficient over time – something that is already happening in
  a lot of ways like in big data – while still being able to do more
  things that they are trained on, like find patterns. I think that's
  where I'd want to use neural networks in the future.

  We don't know, but it will probably happen. That's an interesting
  question to ask. Of course we have AI, which can do interesting things
  without human input. It can recognize the world, find objects and
  images, it understands language, we can make games for it. But if you
  imagine an AI being able to write music, will it even be good enough?
  How much better than human players? How much better can someone else
  do? AI will never really be at its best because it will always have to
  come up with creative ways to use and play it. To ask a question like
  it is good enough is to ignore that it is human. You are only allowed
  to ask good questions

  The software will help, as it already can do. But it will also allow
  more human creators, which is what I think it should do more of to
  bring music to any genre. In terms of composing, it's a matter of
  creativity in humans in general. But how does it achieve that? Let's
  just say that there are fewer humans composing today than ever before.
  Do you remember the day when, by chance, a computer could write lyrics
  to Mozart's Sonata No. 4, or the day when a human wrote lyrics to
  Mozart's La Résistance du Ciel No. 2? That was back in 1960, and it's
  just not true anymore. As we move forward, we are not going to create
  music that human musicians cannot compose. What is your goal as a
  computer music creator?

  At the core of this idea is that music is a collection of sounds – it
  was the very first art form, it was the first music that humans
  evolved, a collection of sounds. Music is not an abstract thing of
  sound, but a collection of sounds. You can't write music the way we
  write and compose books and films, because we write and compose with
  sounds in mind. This is where machine learning comes in. You can read
  some books about how to get better music writers or how to improve the
  quality of their music, but you cannot make a difference with this
  kind of information at all. What does music teach you? To write and
  perform beautifully, to convey emotion accurately, and to create
  lasting and rich relationships with


Teaching a NN to manipulate pseudoRNG over a long time scale? For speedrunning purposes, I am trying to train a neural network to identify human-executable ways to manipulate pseudo-RNG (in Pokemon Red, for the interested). The game runs at sixty frames per second, and the linear-congruential PRNG updates every frame, while many frames are unlikely to be relevant to the manipulation (and so should contain no actions from the neural net). Any given manipulation is likely to last 30sec-2min, and the advancement rate of the PRNG can change depending on location in the game-world.

I have some experience with coding AI/deep-learning. I've made some programs using Multilayer Perceptron and IndRNN approaches. From what I can tell, IndRNN or A3C would be my best bets. I'm not expert enough to know the correct approach, though, or to know if the dimensionality of the problem makes it outright unfeasible.

1) Is this problem reasonably solvable with NN/deep learning?

2) What approach would you recommend to tackle it?

The point of pseudoRNG is to be unmodable and unpredictable, making it hard to train an AI to learn. It would more likely be useful and more efficient to have the equation that the game uses for generation available, so that you can manually make the check, or to just have a list of the loop if the pseudoRNG is based on the time elapsed.


How does reinforcement learning with video data work? Gday,

so my goal is to train an agent to play MarioKart on the Nintendo DS. My first approach (in theory) was to setup an emulator on my pc and let the agent play for ages. But then a colleague suggested to train the agent first on pre recorded humanly played video data, to achieve some sort of base level. And then for further perfection let the agent play for its own with the emulator.

BUT.
I have no clue how training with video data works. E.g. I wonder how to calculate a loss since there is no reward. Or am I getting the intuition wrong?

I would appreciate it if someone could explain this technique to me.

In reinforcement learning, to learn off policy control, you need data on the states, actions and rewards at each time step. If, in addition to a recorded video, you had a recording of controller inputs, and could add reward data by hand, then you could use a standard reinforcement learning method, e.g. DQN. Simply run the DQN training loop as normal, but skip the parts where it acts in the environment, and only train on batches of recorded experience.

With only video data, your options are limited. However, it might still be useful, because a significant part of the challenge is a machine vision task. For a DQN agent, it will need to convert frames from the video (e.g. last 4 frames) into a prediction of the different rewards that it could get depending on which controller buttons are pressed. If you can teach a separate neural network to perform a vision task on relevant video data, it may may help. You could use the learned weights from the first layers of this network as the starting point for your Q values network, and it will likely speed up a DQN figuring out the relationship to its predictions. This sort of task switch following learning is called transfer learning, and is often used in computer vision tasks.

A possibly useful starting task if you have a video, but no controller or reward data, would be to predict the next frame(s) of the video, given say four starting frames (you need more than one so that the neural network can use velocity information). It should be possible to generate the training data using opencv or ffmpeg from your recordings.


Can artificial intelligence take over the world? Will the machine become so powerful that it begins to think to a point where it is more capable than the humans? Notice, I said capable, not powerful. No doubt computers are more powerful at giving us answers faster than human brainpower, but are they more capable? so can artificial intelligence take over the world?.

The world seems to be operating under the "law of maximum intelligence" (I made this up), which is that the species with the most intelligence get to realise its preferences the most strongly. Put another way, the more intelligent you are, the more the world takes the shape of what you want; and this happens through your actions upon the world.

This is tautological if what you mean by "intelligence" is "ability to act well".

For example, both humans and chimps want to be happy, but you get zoos and not chimps paradise, because zoos are optimised to please humans, not chimps.

If you grant this law, then all you need is a "greater than humanity" level of intelligence, and an entity with that level of intelligence should have its preferences met.

As to whever or not this will happen, I’ll quote Sam Harris in this little talk:

  All you have to grant to get your fears [about AI] up and running is
  that:

  We will continue to make progress in hardware and software design
  (unless we destroy ourselves some other way), and
  There’s nothing
  magical about the wetware we have running inside our heads, and that
  an intelligent machine could be build of other material. […]

  Once you’ve granted those two propositions, you’ll now be hard pressed
  to find some hand hold with which to resist yor slide into real
  concern about where this is all going.

To go back to the question: yes if you have an entity/agent that is sufficiently intelligent, and with a goal that would directly or as a side-effect cause it to take over the world.
Note that this is a real concern because of instrumental convergence. Good talk on this topic by Nates Soares.

I also recommend the lesswrong forums: https://wiki.lesswrong.com/wiki/AI_takeoff and Nick Bostrom: https://nickbostrom.com/superintelligence.html (or his book also titled Superintelligence)


Is it possible to compute $P( F \mid S )$ given $P(F \mid S,A)$, $P(F \mid S, \lnot A)$? I have a bayesian network, which has the following data:

$P(S) = 0.07$

$P(A) = 0.01$

$P(F \mid S,A) = 1.0$

$P(F \mid S, \lnot A) = 0.7$

$P(F \mid \lnot S, A) = 0.9$

$P(F \mid \lnot S, \lnot A) = 0.1$

And I'm asked to get $P(F \mid S)$. Is it possible? How can I deduce it?

This is a bit of a puzzle but you can compute a reasonable narrow limit even without knowing whether or not $P(S,A) = P(S) P(A)$.

Start with the contingency table relating $P(S, A)$, $P(S,\neg A)$, $P(\neg S, A)$, $P(\neg S,\neg A)$ to $P(S)$ and $P(A)$ :

$$\begin{array}{cc|c}
P( S,A)& P(\neg S,A) & P(A) \\
P(S,\neg A)& P(\neg S,\neg A) & P(\neg A) \\ \hline
P(S)& P(\neg S) & 1 \\
\end{array} \quad \rightarrow \quad \begin{array}{cc|c}
x& 0.01-x & 0.01 \\
0.07-x& 0.92+x & 0.99 \\ \hline
0.07& 0.93 & 1 \\
\end{array}$$

note that the cells must be between zero and one thus $0 \leq x \leq 0.01$

Then with

\begin{array}{rcrl}P(F|S) &= & & P(F|S,A) P(A|S) + P(F| S,\neg A) P(\neg A|S)\\
&=&& 1.0 \frac{x}{0.07} +  0.7 \frac{0.07-x}{0.07} \\
& = && 0.7 + 4 \frac{2}{7} x  \end{array}

you get

$$0.7 \leq P(F|S) \leq 0.743$$

To solve $P(F|S)$ exactly you need to narrow down $x$ more precisely. Possibilities are:

If you know $P(F)$ then you could use $$\begin{array}{rcrl}P(F) &= & & P(F|S,A) P(S,A) \\ && +& P(F|\neg S,A) P(\neg S,A) \\ &&+& P(F| S,\neg A) P(S, \neg A)\\&&+&P(F|\neg S,\neg A) P(\neg S,\neg A)\\
&=&& 1.0 (x) + 0.7 (0.07 - x) + 0.9 (0.01-x)+0.1(0.92+x) \\
& = && 0.15-0.5 x \end{array}$$
If you know that $S \perp \! \! \! \perp A$ then you can use $x = P(S,A) = P(S)P(A) = 0.0007$


Is normalizing the data a way to improve generalization? There are many known ways to overcome overfitting or make a model generalize better to unseen data.

Here I would like to ask if normalizing/standardizing/similiraizing the train and test data is a plausible approach.

By similarizing I mean making the images look alike by using some function that could be a Neural Network itself. I know that normally one would approach this the opposite way by augmenting and therefore increasing the variation in the training data. But is also possible to improve the model by restricting the variation of the training and test data?

I know that this may not be the best approach and maybe too complicated but I see some use cases where known techniques of preventing overfitting aren't applicable. In those cases, having a network that can normalize/standardize/similarize the "style" of different images could be very useful.

Unfortunately I didn't find a single paper discussing this approach.

Batch Normalization is usually known to speed up the learning process as it makes the weights in the deeper layers more robust. It restricts the distribution of the weights in a particular layer - this video might tend to be useful to what BatchNorm does. This said, batch normalization does have a regularizing effect which does to tend to increase generalization.

Talking about generalization - A focus on regularization would probably be more helpful


Combining mean pooling and max pooling Is it popular or effective to concatenate the results of mean-pooling and max-pooling? To get the invariance of the latter and the expressivity of the former.

I haven't seen it as you describe and I don't think it would be much useful. Pooling layers are being gradually phased out of networks, because they don't seem to be that useful anymore. With the emergence of more and more conv-only architectures, I don't see that likely.


Do we need an explicit policy to sample $A'$ in order to compute the target in SARSA or Q-learning? I would much appreciate if you could point me in the right direction regarding this question about targets for SARSA and Q-learning (notation: $S$ is the current state, $A$ is the current action, $R$ is the reward, $S'$ is the next state and $A'$ is the action chosen from that next state).

Do we need an explicit policy for the Q-learning target to sample $A'$ from? And for SARSA?

I guess this is true for Q-learning since we need to get max Q-value which determines which action $A'$ we'll use for the update. For SARSA, we update the $Q(S, A)$ depending on which action was actually taken (no need for max). Please correct me if I'm wrong.

Q-learning uses an exploratory policy, derived from the current estimate of the $Q$ function, such as the $\epsilon$-greedy policy, to select the action $a$ from the current state $s$. After having taken this action $a$ from $s$, the reward $r$ and the next state $s'$ are observed. At this point, to update the estimate of the $Q$ function, you use a target that assumes that the greedy action is taken from the next state $s'$. The greedy action is selected by the $\operatorname{max}$ operator, which can thus be thought of as an implicit policy (but this terminology isn't common, AFAIK), so, in this context, the greedy action is the action associated with the highest $Q$ value for the state $s'$.

In SARSA, no $\operatorname{max}$ operator is used, and you derive a policy (e.g. the $\epsilon$-greedy policy) from the current estimate of the $Q$ function to select both $a$ (from $s$) and $a'$ (from $s'$).

To conclude, in all cases, the policies are implicit, in the sense that they are derived from the estimate of the $Q$ function, but this isn't a common terminology. See also this answer, where I describe more in detail the differences between Q-learning and SARSA, and I also show the pseudocode of both algorithms, which you should read (multiple times) in order to fully understand their differences.


Training a model to identify certain differences between images? Newbie to CV here so sorry of this is basic. Here's the deal, I have a program that I run many times. and each run I produce a screenshot. I need to compare screenshots from N-1 and N runs and make sure they aren't different in any dramatic way. Of course there are some minor changes like logos and pictures getting updated, etc.
SO far I've used something as simple as absdiff from opencv to highlight the difference regions and then use some sort of threshold to determine whether something passes or not. But I want to make it slightly intelligent but I'm not 100% sure how to proceed. Google hasn't yielded ghe best answers.
Essentially, I want to train the model on many different pairs of images and have the output be binary, yes or no depending on whether it should pass or not. In theory, I should be able to plug in 2 images and based on previous training, it should be able to tell me whether there is significant difference or not. What are some ways I might approach this, particularly with regards to what kinds of models to use.
The requirements here might seem amorphous but that's kinda the nature of the problem. the differences could be, in theory, anything. I am hoping that there will be patterns between different images and that a model would pick up on that. Things like the name of a document is 045 instead of 056 or a logo is slightly updated.

If your image just slightly changes, all you need is the simple algorithm, just find the keyword "k-Nearest Neighbor" or take a look at this link.
To locate where is the difference, you can find the difference by subtracting two images by this script:
def extract_diff(imageA, imageB):
    '''
    Find the different between two image:
        + Input: two RGB image
        + Output: binary image show different between two image
    Assume the different between two image in each channel will be bigger or equal 30
    '''
    subtract = imageB.astype(np.float32) - imageA.astype(np.float32)
    mask_motion = cv2.inRange(np.abs(subtract),(30,30,30),(255,255,255))
    # mask_motion[mask_motion==255] = 1 # scale to 1 to reduce computation
    return mask_motion

And locate where the position with value 255 in the image. Next time you should this kind of question in the stack overflow community, there is a more active community, they can give you a better solution than mine.


Are FFNN (MLP) Lipschitz functions? My question is regarding standard dense-connected feed forward neural networks with sigmoidal activation.

I am studying Bayesian Optimization for hyper-parameter selection for neural networks. There is no doubt that this is an effective method, but I just wan't to delve a little deeper into the maths.

Question: Are neural networks Lipschitz functions?

I'm not an expert in this area, but it would appear to depend on the choice of activation function:

$e^x$ is not Lipschitz continuous. See Analytic functions which are not Lipschitz continuous.
$\tanh(x)$ is.

That said, this paper appears to give some conditions (specifically for dynamic ANNs) for which networks with activation function involving $e^x$ can be Lipschitz continuous, so possibly the above is not the whole story.


What can be deduced about the "algorithm" of backpropagation/gradient descent? On this video

Link to video

a neurologist starts by saying that we do not know how neurons calculate gradients for backpropagation.

At minute 30:39 hes showing faster convergence for "our algorithm", which seems to converge faster than backpropagation.

After 34:36 it goes explaining how "neurons" in the brain are actually packs of neurons.

I do not really understand all that he says, so I infer that those packs of neurons (which seem depicted as a single layer) are the ones who calculate the gradient.
It would make sense if each neuron makes a sightly different calculation, and then each other communicate the difference in results. That would allow to deduce a gradient.

What can be deduced, from the presented information, about the purported "algorithm"?? (From the viewpoint of improving convergence of an artificial neural network).

There are 3 separate issues that are often confounded in Deep Learning and Neuroscience:

Deep Learning is inspired by the way the biological brain works.
Deep Learning is how the biological brain works.
Deep Learning can model how the biological brain works.

Number 1 is accurate. The brain has many layers and many connections. Those principles have informed Deep Learning models.

Number 2 has little evidence to support that claim. The biological brain learns at the cellular level in very different ways than how any Deep Learnings system learns.

Number 3 is a current topic of research. Deep Learning is very good at learning patterns. There are good reasons to believe that Deep Learning can learn patterns in the brain. However, those Deep Learning models will not automatically give insight into the biological processes of the brain.

The video is an example of #1. Inspired by our current understanding of neurobiology, let's build better Deep Learning algorithms. These new algorithms might perform better on machine learning benchmarks. However, these algorithms are not better models of the biological brain. In order to understand the algorithms, the language of biology might not be helpful. It might be better to describe them mathematically.


How could an AI be used to improve the teaching and learning of mathematics? I have been working with AI methods. I am thinking about how my daughter (and also other kids) could learn mathematics with the help of AI. For example, how could an AI be used to show the mistakes that a kid does during the learning path?

Fundraising is probably the most realistic answer with the biggest effect. Similar to simply adding blockchain to the company name helps.

For the question of real application, I should first say that I'm a machine learning engineer who works as a consultant. I had mathematics as a minor subject during bachelor's and Master's degree, which is like 80% of a bachelor's degree in mathematics. Mainly the bachelor's thesis is missing. I also gave private mathematics lessons.

I would distinguish learning mathematics at school from University. It's completely different. However, for this question it does not make much of a difference. It's also not super important that you ask it for math. It's about learning in general.

Machine learning tools can support for several minor tasks:

Figuring out how a symbol is called (I build write-math.com for that as a bachelor's project)
Figuring out what a symbol means
Getting relevant papers / parts of the lecture notes for a given problem
Helping to digitalize stuff for easier sharing / content creation
Learning things by heart: there are some nice apps like Anki which help you to learn arbitrary stuff by heart. The learning algorithm could potentially be improved by AI


Why is the reward function $\text{reward} = 1/{(\text{cost}+1)^2}$ better than $\text{reward} =1/(\text{cost}+1)$? I have implemented a simple Q-learning algorithm to minimize a cost function by setting the reward to the inverse of the cost of the action taken by the agent. The algorithm converges nicely, but there is some difference I get in the global cost convergence for different orders of the reward function. If I use the reward function as:
$$\text{reward} = \frac{1}{(\text{cost}+1)^2}$$
the algorithm converges better (lower global cost, which is the objective of the process) than when I use the reward as:
$$\text{reward} = \frac{1}{(\text{cost}+1)}$$
What could be the explanation for this difference? Is it the issue of optimism in the face of uncertainty?

Reinforcement learning (RL) control maximises the expected sum of rewards. If you change the reward metric, it will change what counts as optimal. Your reward functions are not the same, so will in some cases change the priority of solutions.

As a simple example, consider a choice between trajectories with costs A(0,4,4,4) and B(1,1,1,1). In the original cost formula B is clearly better, with 4 total cost compared with A's cost of 12 - A just has one low cost at the beginning, which I put in deliberately as it exposes the problem with your conversion.

In your two reward formulae:

reward = 1/(cost+1)**2.
  A: 1.0 + 0.04 + 0.04 + 0.04 = 1.12
  B: 0.25 + 0.25 + 0.25 + 0.25 = 1.0

reward = 1/(cost+1).
  A: 1.0 + 0.2 + 0.2 + 0.2 = 1.6
  B: 0.5 + 0.5 + 0.5 + 0.5 = 2.0

So with this example (numbers carefully chosen), maximising the total reward favours A for sum of inverse squares but B for sum of inverses, whilst B should be the clear preference for minimising sum of costs. It is possible to find examples for both of your formulae where the best sum of rewards does not give you the lowest cost.

In your case, if you truly want to minimise total cost, then your conversion to rewards should be:

reward = -cost

Anything else is technically changing the nature of the problem, and will result in different solutions that may not be optimal with respect to your initial goal.


What are the common myths associated with Artificial Intelligence? What are some interesting myths of Artificial Intelligence and what are the facts behind them?

As Artificial Intelligence is rapidly invading in our lives the myths around AI is also fabricating rapidly. Before getting into details one need to get clear off from this myths.

Myth 1: AI will take away our jobs:

Reality: AI is not completely different from other technologies and AI will not take away jobs but AI will change the way we work and helps us to increase the productivity by removing monotonous works.

Myth 2: Artificial intelligence will take over the world:

Reality: AI controlling the world. According to me it will not possible unless we give it that power. AI or robots will assist in our work and helps us to solve some tedious works that are difficult for human to solve easily.

Myth 3: Intelligent machines can learn on their own

Reality: It seems that a Intelligent machine can learn by it own. But the fact is that a AI Engineer or AI specialist should develop the algorithm and feed the machine with datasets and instructions and continuous monitoring should be done and most importantly regular update of software should be done.

Myth 4: Artificial Intelligence, Machine learning and Deep learning all three are same:

Reality: No not at all. To be clear machine learning is a part of AI and deep learning is the subset of ML. All three- AL, ML and DL are different but they are inter related with each other.


How do I turn this formula of the average degree of a graph into Python code? I am working through the textbook "Graph-Based Natural Language Processing and Information Retrieval", where I've got a question on implementation of this first Latex looking formula/algorithm.
Can you help me turn the formula under 1.2 Graph Properties into python code? Yes, I know there are many other languages, but python is more user-friendly so I'm starting there, and will eventually rewrite it into C.
As I read the above node example, sorry the D and E nodes were cut off.
Node A has two outflowing arrows notating it as their head node, and it is the one tail node.
This first sentence references the Graphs:
To traverse from A to B, If A to B value is sufficient (above Nx), go to B. If A to B value is below Nx, go A to C to D to A to B total cost is 5.2+7+1+8 = 21.20 traverse cost, this makes sense.

This sentence refers to the Latex formula in the book.
Then to start the formula calculation, the average degree of a graph "a" is equal to the sum of, one over N, times the sum of the in-degree of vertices? Asserting that the sum is a non zero integer between 1 and N?
Ok, I only loaded one page and hope that's not a TOS violation or causes issue, it's a challenge to find people who understand graph theory.
Let me know what questions you have, but I'm just wanting to get clarification if my understanding is what this page is saying.

Each node is a position in the arrays

values = value of the node
conn = indexes of connected nodes

If its an undirected graph, each node must have all the nodes to which they are attached.
Instead, in directed graphs, only the start node has the index.
For your image:

values = ['A','B','C','D','E']
conn = [[1,2,3],[0,4],[0,3,4],[0,2,3],[1,3]]

Example = 'A' -> 1 ('B') ,2 ('C') ,3 ('D')

def averageGraph(conn):
    if conn.all() != None:
        average = 0
        for node in conn:
            average = average + len(node) #len(node) = nº nodes connected = degree
        return average / len(conn)
    else:
        return None


Is it possible to have a negative output using only ReLU activation functions, but not in the final layer? I know that if you use an ReLU activation function at a node in the neural network, the output of that node will be non-negative. I am wondering if it is possible to have a negative output in the final layer, provided that you do not use any activation functions in the final layer, and all the activation functions in the previous hidden layers are ReLU?

Yes, if there's no activation function in the last layer, the weights could simply be negative there, so the network would multiply a positive value with a negative weight, therefore outputting a negative value.
There is still an activation function, but it is the identity.


Number of LSTM layers needed to learn a certain number of sequences Theoretically, number of units for a LSTM layer is the number of hidden states or the max length of sequences as per my practice.
For example, in Keras:
Lstm1 = LSTM(units=MAX_SEQ_LEN, return_sequences=False);

However, with lots of sequences to train, should I add more LSTM layers? because increasing MAX_SEQ_LEN is not the way as it doesn't help make the network better since the extra number of hidden states isn't useful any more.
I'm considering increasing number of LSTM layers, but how many are enough?
For example, 3 of them:
Lstm1 = LSTM(units=MAX_SEQ_LEN, return_sequences=True);
Lstm2 = LSTM(units=MAX_SEQ_LEN, return_sequences=True);
Lstm3 = LSTM(units=MAX_SEQ_LEN, return_sequences=False);

From my personal experience, the units hyperparam in LSTM is not necessary to be the same as max sequence length. Add more units to have the loss curve dive faster.
And about the number of LSTM layers, trying out a single LSTM layer is a good start point, the model trains better with more LSTM layers.
For example, MAX_SEQ_LEN=10, in Keras:
Lstm1 = LSTM(units=30, return_sequences=True); #Time sequence, to feed to next layer
Lstm2 = LSTM(units=20, return_sequences=True); #Time sequence, to feed to next layer
Lstm3 = LSTM(units=10, return_sequences=False);

Output = Lstm3(Lstm2(Lstm1));
```


What is the difference between a machine learning engineer and deep learning engineer? What is the difference between a Machine Learning Engineer and Deep Learning Engineer and an AI developer? What would be their daily tasks at the office?

Job titles in IT are somewhat arbitrary, and not very consistent even within the same country or general industry. Job titles in software development, especially newer forms of software development such as mobile apps, even more so. The latest trends in Data Science, Machine Learning and "AI" are at the sharp end of that.

There are some common themes - e.g. an AI Researcher is likely to be expected to be up to date with scientific papers and might be expected to produce some (or summaries/applications of same within the work environment) whilst a Deep Learning Engineer might
be expected to be familiar with the ins and outs of provisioning compute resources in Amazon (AWS) or Google (GCP), moving data around for training or inference, with enough coding experience to integrate a TensforFlow model with some already-existing web service. However, none of this is strictly separated and a job advertised as "Data Scientist" for a smaller company might be expected to do both those things and more.

All this means that your best bet is to not take much notice of the job title. Read the job specification carefully. Approach each company with questions about daily tasks, responsibilities and assumed technical skills. Expect in some cases that the companies will be unsure themselves of what the job entails and expecting you to take a mature role in defining your own working relationship with the rest of the company - the more senior and experienced the role being looked for, the more likely this is. Both sides  - prospective employee and prospective employer - may have incorrect assumptions about what is realistic and possible.

TL;DR: It's a bit Wild West out there at the moment, and likely will remain so over the next decade. To reduce variance in your own experience, research job posts carefully and talk in detail to employers about expectations in advance.


What's the difference between biological and artificial evolution? I am trying to understand the difference between biological and artificial evolution. If we look at it in terms of genetics, in both of them, the selection operation is a key term.

What's the difference between biological and artificial evolution?

Biological and artificial evolution work around pretty much the same principles.

Fitness and selection: In biology, the fittest organisms in an ecosystem are more likely to survive long enough to reproduce, passing on their genes in the process. In artificial evolution, our organisms are in fact solutions to our problem, which can be evaluated to determine how good they are (their fitness). We choose ourselves which solutions will be selected for reproduction (there are many ways to do this selection, but what is common among all of them is that the fittest solutions have a higher chance of being selected).

Crossover: In biology, an organism inherits a portion of each parent's genes, so is a sort of genetic hybrid of both parents. For artificial evolution, a new solution (a "child" solution) will inherit part of its parent's solutions (we take a partial solution from each parent, and glue those partial solutions together to construct a new solution).

Mutation: In nature mutations often occur at birth and this is why there are many different species. Harmful mutations make the individual less likely to survive long enough to pass them on to children, and in contrast helpful mutations make it more likely that the individual will survive long enough to pass them unto children. The same can be said for artificial evolution: A mutation randomly changes a small part of the solution, and if it makes that solution fitter, then that solution has a higher chance of being selected for reproduction.


Running 2 NEAT nets on the same observations So i have been playing around with neat-python. I made a program, applying neat, to play pinball on the Atari 2600. The code for that can be found in the file test2.py here

Now based on that, I would like to do the same, but on a 2 player game. I have already set up the environment to play a 2 player game, which PONG using OpenAI Retro.

What I have no clue how to do, is run 2 nets at the same time, on the same observation. The way that neat-python works, is you get the observation from a single function that goes through each genome and runs the environment.

How would you create 2 eval_genome functions that can take in the same observation real-time? This means that they train based off of the same images and environmenrs.

Help?

I'm not familiar with neat-python, but I have implemented NEAT to do openai tasks. If there is a class for initializing a population, you could just use that and have 2 objects like population1 and population2 and call them in the same loop.


What should the initial UCT value be with MCTS, when leaf's simulation count is zero? Infinity? I am implenting a Monte Carlo Tree Search algorithm, where the selection process is done through Upper Confidence Bound formula:
def uct(state):
        log_n = math.log(state.parent.sim_count)
        explore_term = self.exploration_weight * math.sqrt(log_n / state.sim_count)
        exploit_term = (state.win_count / state.sim_count)

        return exploit_term + explore_term

I have trouble however choosing the initial value for UCT, when the sim_count of the node is 0. I tried with +inf (which would be appropriate as approaching lim -> 0 from the positive side would give infinity), but that just means the algorithm will be always choosing an unexplored child.
What would you suggest as a initial value for the uct?
Thank you in advance!

Assigning a value of $\infty$ to unvisited nodes is indeed the "default" or most basic choice, and it indeed ensures that the search never visits a node for a second time if it also still has siblings that have not had any visits. But many other kinds of values have been tried in the literature too.
Gelly and Wang, in "Exploration exploitation in Go: UCT for Monte-Carlo Go" referred to the parameter as "First-play Urgency" (FPU), and indeed really treated it as a hyperparameter; they simply tried various different values and some were found to work better than others.
Other more specific values that you may want to consider (but in most cases we can't really say much about which one will be better than which other ones, without empirical evaluations) are:

The value of a loss; treating nodes that have not had any visits as losing nodes is probably the most "pessimistic" initialisation you could pick. It's not very clearly described in the papers, but there is some evidence out there that this was used by AlphaGo Zero and AlphaZero. Those are not simply vanilla UCTs though, those programs had very strong trained Deep Neural Networks to make additional recommendations for actions. If you don't have such strong neural networks to introduce additional biases in your selection, I would not recommend treating unvisited nodes as losing nodes.
The value of a draw; treating unvisited nodes as draws means that you may prioritise re-visiting nodes that look like winning nodes, but you won't prioritise re-visiting nodes that look like losing nodes.
The value of a win; this probably produces similar behaviour in practice as a value of $\infty$
The average value backpropagated into the parent (correcting for differences in player-colour-to-move); you may prioritise re-visiting nodes that perform better than average in this subtree, but won't prioritise re-visiting nodes that perform worse than average in this subtree


How to replicate legacy systems with machine learning? Let's suppose that we have a legacy system in which we don't have the source code and this system is on a mainframe written in Cobol. Is there any way using machine learning in which we can learn from the inputs and outputs the way the executables work? Doing this analysis could lead to develop some rest / soap webservice that can substitute the legacy system in my opinion.

Let's assume from the outset that the space of inputs is too large to allow exhaustive tabulation.

The essential issues when applying ML are that the program being modelled is likely in general to:

Be discrete, i.e. operate (at least in part) on integer, boolean or categorical variables.
Contain various conditional/looping constructs (if/while/for etc).
Have side-effects that affect other parts of the program (e.g. non-local variables) or world state (e.g. writing to a file).

These pose obstacles for ML methods such as ANN. The ML approach which is most immediately compatible with these issues is Genetic Programming (GP).
A recent specialisation of GP that is specifically concerned with the transformation of existing software systems is Genetic Improvement (GI).

However neither GP/GI (nor any other current ML technique) is a 'silver bullet' here:

Despite decades of research, GP still works best at synthesizing relatively small functions, certainly not entire legacy programs.
Because it is only possible to train on a very small subset of a program's input space, there is little guarantee that the program will generalize to inputs it hasn't been trained on.
How will the success of side effects be formally determined for training purposes?

Some of these issues could be addressed to some degree if the program has a comprehensive test suite, but replacing an entire nontrivial program is not likely anytime soon. Replacing smaller parts of the program that have good unit tests is more realistic goal.

Here is a case study showing how GI was successfully used to fix errors in the implementation of Apache Hadoop, by operating only on the program binary.


Denoising Images When Training a Classification Model Suppose you have a binary outcome variable and have some training data (10,000 images in jpg format). Also you have a test set of say 11,000 images. If we want to train a classification model and want to improve the image quality (denoise the images), should we do it for every image in the training set? Likewise, should we do it for very image in the test set? Or should the images in training set randomly be chosen to be denoised (likewise in the test set)?

You can denoise a certain fraction of images (preferably 0.25-0.3) randomly for each epoch.
Adding Gaussian noise to images gives better results.Refer this : Link


Keras DQN Model with Multiple Inputs and Multiple Outputs I am trying to create a DQN agent where I have 2 inputs: the agent's position and a matrix of 0s and 1s.
The output is composed of the agent's new chosen position, a matrix of 0s and 1s (different from the input matrix), and a vector of values.
The first input is fed to an MLP network, the second input (matrix) is fed to a convolutional layer, then their outputs are fed to a FC network, or at least that's the idea.
This is my attempt so far, having this tutorial as a reference.
Here is the code:
First, create the MLP network
def create_mlp(self, arr, regress=False): # for the position input
        # define MLP network
        print("Array", arr)
        model = Sequential()
        model.add(Dense(env.rows * env.cols, input_shape=(len(arr)//2, len(arr)), activation="relu"))
        model.add(Dense((env.rows * env.cols)//2, activation="relu"))

        # check to see if the regression node should be added
        if regress:
            model.add(Dense(1, activation="linear"))

        # return our model
        return model

Then, the CNN
def create_cnn(self, width, height, depth=1, regress=False): # for the matrix
        # initialize the input shape and channel dimension
        inputShape = (height, width, depth)
        output_nodes = 6e2

        # define the model input
        inputs = Input(shape=inputShape)

        # if this is the first CONV layer then set the input
        # appropriately
        x = inputs

        input_layer = Input(shape=(width, height, depth))
        conv1 = Conv2D(100, 3, padding="same", activation="relu", input_shape=inputShape) (input_layer)
        pool1 = MaxPooling2D(pool_size=(2,2), padding="same")(conv1)
        flat = Flatten()(pool1)
        hidden1 = Dense(200, activation='softmax')(flat) #relu

        batchnorm1 = BatchNormalization()(hidden1)
        output_layer = Dense(output_nodes, activation="softmax")(batchnorm1)
        output_layer2 = Dense(output_nodes, activation="relu")(output_layer)
        output_reshape = Reshape((int(output_nodes), 1))(output_layer2)
        model = Model(inputs=input_layer, outputs=output_reshape)

        # return the CNN
        return model

Then, concatenate the two
def _build_model(self):
        # create the MLP and CNN models
        mlp = self.create_mlp(env.stateSpacePos)
        cnn = self.create_cnn(3, len(env.UEs))

        # create the input to our final set of layers as the *output* of both
        # the MLP and CNN
        combinedInput = concatenate([mlp.output, cnn.output])

        # our final FC layer head will have two dense layers, the final one
        # being our regression head
        x = Dense(len(env.stateSpacePos), activation="relu")(combinedInput)
        x = Dense(1, activation="linear")(x)

        # our final model will accept categorical/numerical data on the MLP
        # input and images on the CNN input, outputting a single value
        model = Model(inputs=[mlp.input, cnn.input], outputs=x)

        opt = Adam(lr=self.learning_rate, decay=self.epsilon_decay)
        model.compile(loss="mean_absolute_percentage_error", optimizer=opt)

        print(model.summary())

        return model

I have an error:
A `Concatenate` layer requires inputs with matching shapes except for the concat axis. Got inputs shapes: [(None, 32, 50), (None, 600, 1)]

The line of code that gives the error is:
combinedInput = concatenate([mlp.output, cnn.output])

This is the MLP summary

And this is the CNN summary

I'm a beginner at this, and I'm not where my mistakes are, the code does not work obviously but I do not know how to correct it.

Firstly, concatenate only works on identical output shape of the axis. Otherwise, the function will not work.
Now, your function output size is (None, 32, 50) and (None, 600, 1). Here, '32' and '600' must be same when you want to concatenate.
I would like to suggest some advice based on your problem. You can flatten both of them first and then concatenate. Because you need to flatten feature to use dense layer later.
def create_mlp(self, arr, regress=False):
        # define MLP network
        print("Array", arr)
        model = Sequential()
        model.add(Dense(env.rows * env.cols, input_shape=(len(arr)//2, len(arr)), activation="relu"))
        model.add(Dense((env.rows * env.cols)//2, activation="relu"))
        **model.add.flatten() ### shape = (None, 1600)**
        # check to see if the regression node should be added
        if regress:
            model.add(Dense(1, activation="linear"))
        # return our model
        return model

And just remove the reshape layer in create_cnn function.
(output shape should be = (None, 600)).
then concatenate two model
combinedInput = concatenate([mlp.output, cnn.output])  ## output shape =(None, 2200)
Later you can just use Dense layer as your code. I don't know how can you used dense (next to concatenate layer) without flatten the feature in create_mlp function.
Your code should work this way.
You can read this simple one for better understanding.


Examples of time-varying graph-structured data in real world I'm looking for examples of time-varying graph-structured data for time-varying graph CNNs. First, I came up with the idea of infection network. Is there anything more? If possible, I want data that can be easily obtained online.

You can take a look at traffic data for example if you follow
link1, link2 you can find 3 publicly available traffic datasets which are already preprocessed. You cold also look at air quality datasets offered by the government link3


Can Google's patented ML algorithms be used commercially? I just find that Google patents some of the widely used machine learning algorithms. For example:

System and method for addressing overfitting in a neural network (Dropout?)
Processing images using deep neural networks
Methods and apparatus for reinforcement learning (Q-Learning?)

Is that mean I can't use those algorithms commercially?

Can you use them commercially?

Yes.

Is Google able to sue you any time they want?

Yes.

Will they do that...

Probably not.

Google isn't a known patent bully, I would give them the benefit of the doubt in this kind of situation and say, unless you start really giving them real trouble, they wouldn't do anything. Some companies/people know an idea can really be used for good and patent it to protect its use not to inhibit its use. By patenting the idea and setting a precedent of not suing they are effectively allowing everyone to benefit. Maybe in the future Google cloud, Azure and Amazon web services will lose some money in a legal battle, but I doubt you personally will be hit with a lawsuit.


What is this algorithm? Is it a variant of Monte-Carlo Tree Search? I'm using a Neural Network as an agent in a simple car racing game. My goal is to train the network to imitate a brute-force tree search to an arbitrary depth.
My algorithm goes something like the following:

The agent starts with depth 0
Generate a bunch of test states
For each test state s:

For each action a:

s' = env.step(s, a)
for x in range(agent.depth): s' = env.step(s', agent.predict(s'))
calculate reward at state s'

Set the label for test state s as whichever action a produced the highest reward

Train the agent using the test states and labels, and increment agent.depth
Loop until desired depth

The idea is that an agent trained in this way to depth N should produce output close to a brute-force tree search to depth N...so by using it to play out N moves, it should be able to find me the best final state at that depth. In practice, I've found that it performs somewhere between N and N-1 (but of course it never reaches 100% accuracy).
My question is: what is the name of this algorithm? When I search for tree search with playout, everything talks about MCTS. But since there's no randomness here (the first step is to try ALL actions), what would this be called instead?

In short it looks like you have constructed a valid reinforcement learning method, but it does not have much in common with Monte Carlo Tree Search. It may have some weaknesses compared to more established methods, that means it will work better in some environments rather than others.
Your approach may be novel, in that you have combined ideas into a method which has not been used in this exact form before. However, it is following the principles of general policy improvement (GPI):

Estimate action values of a current policy.

Create a new policy by maximising action choices with respect to latest estimates.

Set current policy to new policy and repeat.

Your method covers exploration with a deterministic policy by sweeping through a list of state and action pairs. This resembles policy iteration, or perhaps exploring starts in Monte Carlo control. It evaluates each state/action pair by following the current policy up to a time step horizon. This has some weakness, but it may work well enough in some cases.
The depth iteration is more complex to analyse. It is not doing what you suggest - i.e. it does not make the whole algorithm equivalent somehow to a tree search. Technically the value estimates with a short horizon will be poor, biased estimates of true value in the general case, but on the plus side they may still help differentiate between good and bad action choices in some situations. It may even help as a form of curriculum learning by gathering and training on data about immediately obvious decisions first (again I expect that would be situational, not something you could rely on).
Overall, although you seem to have found a nice solution for your racing game, I think your method will work unreliably in a general case. Environments with stochastic state transitions and rewards would be a major problem, and this is not something you could fix without major changes to the algorithm.
I could suggest that you to try one or more standard published methods, such as DQN, A3C etc, and compare them with your approach on the same environments in different ways. E.g. how much time and computation it takes to train to an acceptable level, or how close to optimal each method can get to.
The main comparison with Monte Carlo Tree Search is that you evaluate each state, action pair with a kind of rollout. There is lots more going on in MCTS that you are not doing though, and the iteration with longer rollouts each time is not like anything in MCTS and does not compensate for missing parts of MCTS in your algorithm.


Will Q-learning converge to the optimal state-action function when the reward periodically changes? Imagine that the agent receives a positive reward upon reaching a state . Once the state  has been reached the positive reward associated with it vanishes and appears somewhere else in the state space, say at state ′. The reward associated to ′ also vanishes when the agent visits that state once and re-appears at state . This goes periodically forever. Will discounted Q-learning converge to the optimal policy in this setup? Is yes, is there any proof out there, I couldn't find anything.

No, it will not converge in the general case (maybe it might in extremely convenient special cases, not sure, didn't think hard enough about that...).

Practically everything in Reinforcement Learning theory (including convergence proofs) relies on the Markov property; the assumption that the current state $s_t$ includes all relevant information, that the history leading up to $s_t$ is no longer relevant. In your case, this property is violated; it is important to remember whether or not we visited $s$ more recently than $s'$.

I suppose if you "enhance" your states such that they include that piece of information, then it should converge again. This means that you'd essentially double your state-space. For every state that you have in your "normal" state space, you'd have to add a separate copy that would be used in cases where $s$ was visited more recently than $s'$.


How can I reduce combinatorial explosion in an MCTS-like algorithm for program induction? I'd like to develop an MCTS-like (Monte Carlo Tree Search) algorithm for program induction, i.e. learning programs from examples.

My initial plan is for nodes to represent programs and for the search to expand nodes by revising programs.

Many of these expansions revise a single program: randomly resample a subtree of the program, replace a constant with a variable, etc. It looks straightforward to use these with MCTS.

Some expansions, however, generate a program from scratch (e.g. sample a new program). Others use two or more programs to generate a single output program (e.g. crossover in Genetic Programming).

These latter types of moves seem nonstandard for vanilla MCTS.

One idea I've had is to switch from nodes representing programs to nodes representing tuples of programs. The root node would represent the empty tuple $()$, to which expansions could be applied only if they can generate a program from scratch. The first such expansion would produce some program $p$, so the root would now have child $(p)$. The second expansion would produce $p'$, so the root would now also have child $(p')$ as well as the pair $(p, p')$. Even assuming some reasonable restrictions (e.g. moves can use at most 2 programs, pairs cannot have identical elements, element order doesn't matter), the branching factor will grow combinatorially.

What techniques from the MCTS literature (or other literatures) might reduce the impact of this combinatorial explosion?

I think you may get some inspiration from the work on deterministic environments by Javier Segovia et al. See their paper Computing programs for generalized planning using a classical planner (2019).
If you don't have access to Elsevier's papers, I recommend that you check out the first author's profile on dblp.org. From there you will find links to open access versions of the conference paper that led up to the publication above.


How can I use Neural Network in motion identification I'm quite new to image processing and AI. But I have the expertise to create a network that can be used in object detection and recognition. Most of the time I've used ANN or Naive Bayes.

Now, I want to develop a method of action recognition, something like identifying whether one is jogging, running or walking by applying ANN. However, I really don't have idea how the sequence of frames can be classified.

In static image, segmentation and feature extraction is easy. But in regard to a moving image, I'm unsure of the approach.

Thanks in advance!

LSTM's provide a simple way to process sequential data (assuming all video sequences have the same number of frames and resolution).


Could artificial general intelligence harm humanity? Could advanced AI decide to kill all humans?

This is a very, very controversial topic. But here's my opinion on the topic:

AI won't harm humanity (at least, to the level that a lot of people think it will).

As Kartik mentioned in his answer, AGI (Artificial General Intelligence) is the concept of AI that's become so "generally" intelligent, that it can do everything we as humans can, and more, but do it better, more efficiently, more accurately, you name it.

I'd say that AGI is (so far) a fictional scenario. Why? Because of the physical limitation of computing that wouldn't let us get there. In order to simulate natural, physical intelligence, you wouldn't be able to use gradient descent, you would need to simulate physical interactions of neurons. Simulating even a single molecule on the fastest supercomputers takes a very long time, because the overhead introduced with taking the real, physical world, transposing it onto mathematics, and running those mathematics on electronics, are too much. So simulating the complete working of just a small cluster of neurons would be almost infeasible.

Then you might ask: "what if we were to get enough computing power in the future?". It's a valid question, however, there's one problem: we're already (possibly) quite close to the physical limit of how small we can get our transistors before Quantum Tunnelling kicks in.

So I'd say it's impossible because of a few key reasons:

Current algorithms are intrinsically guided by human-generated data (even reinforcement learning algorithms need human-engineered rewards to understand what's good and what's bad). Therefore, there's no such thing as a self-guided objective for machine learning today.
Computing power isn't there yet, and most probably, will never get near the level required for AGI.
Humans have an innate bias to "humanize" non-living entities. Language, for example, is a great representation of the way our minds work on a fundamental, subconscious level. When we say "my computer isn't co-operating with me", or something along those lines, we don't realize that computers are big calculators with a screen and lots of predefined operations to run.

One thing I don't disagree with: Machine Learning will replace a lot of jobs - but not nearly as many as you may think! It'll take time - if Tesla eventually comes out with a fully-autonomous semi-truck, it's not like the whole industry will adopt to it overnight. Humans, and our genome, will have time to adapt and evolve overtime. We'll free up more people, over time, to get into fields like healthcare, education, and entertainment. When we don't have the option to get jobs like cashiers, drivers, etc., we'll be forced to get more advanced jobs.

This is just how technology has worked, and will continue to work for as long as humanity exists.

People, however, can use AI/ML for negative purposes and bad things. Face swapping, automated social engineering, etc., are all great examples of this. But with powerful unregulated/illegal technology, comes even more powerful regulated/legal technology.

As pointed out in the comments, transistors are not the only solution. There are alternatives like Quantum Computing, that enable you to remove that mathematical overhead, and directly run your operations on qubits. However, there's still a lot of work to be done to see if technology like AGI or self-guided machine learning is possible even with quantum computers.

None of this is certain - anything I've just mentioned can change in the next few weeks, months, or years.


Difficulty understanding Keras LSTM fitting data I'm try to train a RNN with a chunk of audio data, where X and Y are two audio channels loaded into numpy arrays.  The objective is to experiment with different NN designs to train them to transform single channel (mono) audio into a two channel (stereo) audio.

My questions are:

Do I need a stateful network type, like LSTM? (I think yes.)
How should I organize the data, considering that there are millions of samples and I can't load into memory a matrix of each window of data in a reasonable time-span?

For example if I have an array with: [0, 0.5, 0.75, 1, -0.5, 0.22, -0.30 ...] and I want to take a window of 3 samples, for example.  I guess I need to create a matrix with every sample shift like this, right?

[[0.00, 0.50, 0.75]
 [0.50, 0.75, 1.00]
 [0.75, 1.00,-0.50]
 [1.00,-0.50, 0.22]]

Where is my batch_size? Should I make the matrix like this per each sample shift? Per each window? This may be very memory consuming if I intend to load a 4 min song.

Is this example matrix a single batch? A single sample?

Yes, intuition says that RNNs like LSTM or GRU will work better in your case, because predicted values might depend on input patterns corresponding to much earlier time intervals.
There is no reason to create samples shifted by a single measurement because many of the samples will contain pretty much the same information for your model. A viable approach is to shift by sample size. Keeping some overlap between samples is feasible as well.

Keep in mind that as a general rule for processing audio data, it makes sense to convert raw audio data into vectors representing audio spectrum before feeding it into LSTM RNN (see this video for e.g. https://www.coursera.org/learn/nlp-sequence-models/lecture/sjiUm/speech-recognition).

Batch size is different from sample or window size in your case.


Which loss function should I use for binary classification? I plan to create a neural network using Python, Keras and TensorFlow. All the tutorials I have seen so far are concerned with image recognition. However, the goal of my program would be to take in 10+ inputs and calculate a binary output (true/false) instead. What would you suggest me to learn for this specific purpose?

There are several loss functions that you can use for binary classification. For example, you could use the binary cross-entropy or the hinge loss functions. See, for example, the tutorials Binary Classification Tutorial with the Keras Deep Learning Library (2016) and How to Choose Loss Functions When Training Deep Learning Neural Networks (2019) by Jason Brownlee. Have also a look at Keras documentation of its available loss functions.


Why do we average gradients and not loss in distributed training? I'm running some distributed trainings in Tensorflow with Horovod. It runs training separately on multiple workers, each of which uses the same weights and does forward pass on unique data. Computed gradients are averaged within the communicator (worker group) before applying them in weight updates. I'm wondering - why not average the loss function across the workers? What's the difference (and the potential benefits) of averaging gradients?

The whole idea behind those distributed optimization methods is that data should be local in every node/worker. Thus, if you only send the loss value to the central node, this node can't compute the gradients of this loss, and thus can't do any training. However, if you don't want to send gradients, a family of distributed optimization algorithms called consensus-based optimization can be used to only send the local weight of the model to neighbouring nodes, and those nodes use their local gradient and the models from their neighbours to update their local models.


Is there a way that helps me to architect my CNN fundamentally before training? While we train a CNN model we often experiment with the number of filters, the number of convolutional layers, FC layers, filter size, sometimes stride, activation function, etc. More often than not after training the model once, it is just a trial & error process.

Is there a way that helps me to architect my model fundamentally before training?
Once I train model, how do I know which among these variables (number of filters, size, number of convolutional layers, FC layers) should be
changed - increased or decreased?

P.S. This question assumes that data is sufficient in volume and annotated properly and still accuracy is not up to the mark. So, I've ruled out the possibility of non-architectural flaws for the question.

See Analysis and Optimization of Convolutional Neural Network Architectures (my master's thesis).

  Is there a way that helps me to architect my model fundamentally before training?

Yes, but the architecture learning approaches are computationally intensive. When I wrote my masters thesis it was ~250,000 USD to run the Google experiment. Meanwhile, there seem to be more efficient methods, e.g. https://autokeras.com/

See Chapter 3.

  Once I train model how do I know which among these variables (number of filters, size, number of convolutional layers, FC layers) should be changed - increased or decreased?

See Chapter 2.5 for some approaches. But there is no silver bullet / no clear answer to this question.


How to solve the problem of too big activations when using genetic algorithms to train neural networks? I am trying to create a fixed-topology MLP from scratch (with C#), which can solve some simple problems, such as the XOR problem and MNIST classification. The network will be trained purely with genetic algorithms instead of back-propagation.
Here are the details:

Population size: 50
Activation function: sigmoid
Fixed topology
XOR: 2 inputs, 1 output. Tested with different numbers of hidden layers/nodes.
MNIST: $28*28=784$ inputs for all pixels, will be either ON(1) or OFF(0). 10 outputs to represent digits 0-9
Initial population will be given random weights between 0 and 1
10 "Fittest" networks survive each iteration, and performs crossover to reproduce 40 offspring
For all weights, mutation occurs to add a random value between -1 to 1, with a 5% chance

With 2 hidden layers of 4 and 3 neurons respectively, XOR managed to achieve 97-99.9% accuracy in around 100 generations. Biases were not used here.
However, trying out MNIST revealed a pretty glaring issue - the 784 inputs; a large increase of nodes compared to XOR, multiplied with weights and added up results in HUGE values of 50 to even 100, way beyond the typical domain range of the activation function (sigmoid).
This just renders all layers' outputs as 1 or 0.99999-something, which breaks the entire network. Also, since this makes all individuals in a population extremely similar to one other, the genetic algorithm seems to have no clue on how to improve. The crossover will produce an offspring almost identical to its parents, and some lucky mutations are simply ignored by the sheer amount of other neurons!
What can be a viable solution to this?
It's my first time studying NNs, and this is really challenging.

Your inputs should stay in a low range. Ideally for neural networks, the inputs are normalised to mean 0, standard deviation 1. I suspect this applies equally well to GA-driven NNs as gradient-driven ones.

Your weights should be both positive and negative.

In addition, once trained, they tend to follow a certain size distribution. It helps if you start with values within that range. This is often called Xavier or Glorot initialisation. Basically, if your number of inputs to a layer is n_inputs and number of outputs is n_outputs, you will have n_inputs * n_outputs weights, and if you initialise them with a random number generator, then you should use a multiplier something like this:

w = (rand() - 0.5) * sqrt(6/(n_inputs + n_outputs))

Note that is an addition inside the square root, you don't take the total number of weights.

As you are using a GA, you may want to use some form of clipping or normalisation to prevent mutations from drifting too far away from these useful weights. I'm not entirely sure what would be best, as usually I would use back propagation to train a large network. Potentially a maxnorm regularisation would help - choose a value (maybe even make it adjustable via mutation) that the norm of weight matrix in each layer should not exceed (separately for each layer, or one overall value, up to you), and if any learning step creates a network with too high a norm, scale it down. The L2 norm, sqrt(sum_of_squared_weights) is usual choice for maxnorm regularisation, and this works well with backpropagation-based supervised learning.