Attempting to solve a optical character recognition task using a feed-forward network I am doing some experimentation on neural networks, and for that I am trying to program a plain OCR task. I have learned CNNs are the best choice ,but for the time being and due to my inexperience, I wanna go step by step and start with feedforward nets.

So my training data is a set of roughly 400 16*16 images extracted from a script that draws every alphabet char in a tiny image for a small set of fonts registered in my computer.

Then the test data set is extracted from the same procedure, but for all fonts in my computer.

Well, results are quite bad. Get an accuracy of aprox. 45-50%, which is very poor... but that's not my question.

The point is that I can't get the MSE below 0.0049, no matter what hidden layer distribution I apply to the net. I have tried with several architectures and all comes down to this figure. Does that mean the net cannot learn any further given the data?

This MSE value however throws this poor results too.

I am using Tensorflow API directly, no keras or estimators and for a list of 62 recognizable characters these are examples of the architectures I have used: [256,1860,62] [256,130,62] [256,256, 128 ,62] [256,3600,62] ....

But never get the MSE below 0.0049, and still results are not over 50%.

Any hints are greatly appreciated.

If you are serious about OCR systems and not just trying to learn CNN, start with a known good OSS product like tesseract. OCR some 1200 DPI grayscale documents, reverse engineer it to some degree so that you understand the design, join the development team, and pick up a ticket or two. Once you get into pull requests for a real system that is in constant use for a while, you will be an expert. Starting from scratch would make sense after you are an expert.


Context-based gap-fill face posture-mapper GAN These images are handmade, not auto-generated like they will be in production. Apologies for inaccuracies in the graph overlay.

I am trying to build an AI like that displayed in the diagram: when given a training set of images with their corresponding node maps of face/nose posture, and an image with a missing section (just a gap) with a node map, I would like it to reconstruct the initial image. My thoughts immediately went to GANs for this, but after some searching, the closest I could find were:

Face recreation without context/not filling gaps, just following pose (DeepFake)
Filling gaps in images, but with no node reference
Filling gaps from reference drawings/mappings, but with no way to provide sample images

I would like to hear about any implementations of such an algorithm, if possible optimised for faces, and if none exists, I would like to hear of how I would go about altering the generator of the GAN to work with the context/gap-fill bit (e.g a paper which talks about this idea, but doesn't implement it). Any guidance on the NN that is best for this type of task is also appreciated.

I believe you may want to use a Sum Product Network for this task. SPNs are the state-of-the-art approach for face completion, and there are several more recent papers on this topic since the original above.

Importantly, the SPN paper also covers other approaches that work well for this task. If lower-resolution results are acceptable for your task, PCA with 100 or more components works surprisingly well. If your dataset is very large, a nearest-neighbor approach can work decently too.


Research for misbehavior detection in WiFI networks I'm looking for research which discusses misbehavior detection in public internet access networks using ANN approaches.

So it can be used by ISP to detect suspicious users connected to their network.

One popular technique for doing this is to use Artificial Immune Systems, an evolutionary computation approach which maintains a population of pattern detectors.

Here is a survey paper.


How to implement a variable action space in Proximal Policy Optimization? I'm coding a Proximal Policy Optimization (PPO) agent with the Tensorforce library (which is built on top of TensorFlow).
The first environment was very simple. Now, I'm diving into a more complex environment, where all the actions are not available at each step.
Let's say there are 5 actions and their availability depends on an internal state (which is defined by the previous action and/or the new state/observation space):

2 actions (0 and 1) are always available
2 actions (2 and 3) are only available when the internal state is 0
1 action (4) is only available when the internal state is 1

Hence, there are 4 actions available when the internal state is 0 and 3 actions available when the internal state is 1.
I'm thinking of a few possibilities to implement that:

Change the action space at each step, depending on the internal state. I assume this is nonsense.

Do nothing: let the model understand that choosing an unavailable action has no impact.

Do almost nothing: impact slightly negatively the reward when the model chooses an unavailable action.

Help the model: by incorporating an integer into the state/observation space that informs the model what's the internal state value + bullet point 2 or 3

Are there other ways to implement this? From your experience, which one would be the best?

The most straightforward solution is to simply make every action "legal", but implementing a consistent, deterministic mapping from potentially illegal actions to different legal actions. Whenever the PPO implementation you are using selects an illegal action, you simply replace it with the legal action that it maps to. Your PPO algorithm can then still update itself as if the illegal action were selected (the illegal action simply becomes like... a "nickname" for the legal action instead).

For example, in the situation you describe:

  2 actions (0 and 1) are always available
  2 actions (2 and 3) are only available when the internal_state == 0
  1 action (4) is only available when the internal_state == 1

In cases where internal_state == 0, if action 4 was selected (an illegal action), you can always swap it out for one of the other actions and play that one instead. It doesn't really matter (theoretically) which one you pick, as long as you're consistent about it. The algorithm doesn't have to know that it picked an illegal action, whenever it picks that same illegal action in the future again in similar states it will consistently get mapped to the same legal action instead, so you just reinforce according to that behaviour.

The solution described above is very straightforward, probably the most simple to implement, but of course it... "smells" a bit "hacky". A cleaner solution would involve a step in the Network that sets the probability outputs of illegal actions to $0$, and re-normalizes the rest to sum up to $1$ again. This requires much more care to make sure that your learning updates are still performed correctly though, and is likely a lot more complex to implement on top of an existing framework like Tensorforce (if not already somehow supported in there out of the box).

For the first "solution", I wrote above that it does not matter "theoretically" how you choose you mapping. I absolutely do expect your choices here will have an impact on learning speed in practice though. This is because, in the initial stages of your learning process, you'll likely have close-to-random action selection. If some actions "appear multiple times" in the outputs, they will have a greater probability of being selected with the initial close-tor-andom action selection. So, there will be an impact on your initial behaviour, which has an impact on the experience that you collect, which in turn also has an impact on what you learn.

I certainly expect it will be beneficial for performance if you can include input feature(s) for the internal_state variable.

If some legal actions can be identified that are somehow "semantically close" to certain illegal actions, it could also be beneficial for performance to specifically connect those "similar" actions in the "mapping" from illegal to legal actions if you choose to go with that solution. For example, if you have a "jump forwards" action that becomes illegal in states where the ceiling is very low (because you'd bump your head), it may be better to map that action to a "move forwards" action (which is still kind of similar, they're both going forwards), than it would be to map it to a "move backwards" action. This idea of "similar" actions will only be applicable to certain domains though, in some domains there may be no such similarities between actions.


How to apply hyperparameter optimization on Monte Carlo Tree Search? I have a basic MCTS agent for the game of Hex (a turn based game). I want to tune the parameters of UCT (the Cp parameter) and the number of rollouts parameter.

Where do I have to begin? The problem is that the agent is smart enough to always win if it plays first against another agent. So I don't know how to do the evaluation of each pair of hyperparameters.

If anyone has any ideas let me know.

Since no one answers, I will say the approach I used.
It was inspired by the successive halving algorithm. The concept is that we find good performing hyperparameters for one agent by playing manually against it. Then, we put the agent we want to tune to play against it. We assign different configurations. All of these configurations are tried for a predefined number of games per configuration. We keep the best performing half, and then go to the next round with the half we kept and repeat this process. The last configuration that remains is the best performing configuration.


What's done towards AI learning new ways of learning? Most (all I know) machine learning systems use a fixed set of data input channels and processing algorithms, only expanding underlying dataset processed by these; they obtain new data but only from predefined sources, and use only their fixed, built-in capacity to process it, possibly tweaking parameters of the algorithm (like weights of neural network nodes) but never fundamentally changing the algorithm.

Are there systems - or research into creating these - that are able to acquire "from out there" new methods of obtaining data and new ways to process it for results? Expand not just passive data set to "digest it" by active but static algorithm, but make the algorithm itself self-expanding - be it in terms of creating/obtaining new processing methods for own data set, and creating/obtaining new methods of acquiring that data (these methods)?

I think the closest thing is building up knowledge using predictions like in the Horde Architecture. Research about what are good predictions to make is on going at the University of Alberta, Canada but the Horde architecture has the potential to ask new questions and generate new data based on the answers to those questions in the form of predictions.

One example is having a robot with a bump sensor and servos and asking questions like

  Will rotating the servos CW for 90 degrees make the bump sensor turn on?

This question can be phrased and it's answer estimated mathematically in the form of a General Value Function as defined in the paper.

Asking different questions based on predictions like this one is where the power comes from.


Advice on creating a new environment using OpenAI Gym I'm looking for some general advice here before I dive in.

I'm interested in creating a new environment for OpenAI gym to provide some slightly more challenging continuous control problems than the ones currently in their set of Classic Control environments.  The intended users of this new environment are just me and members of a meetup group I am in but obviously I want everyone to be able to access, install and potentially modify the same environment.

What's the easiest way to do this?

Can I simply import and sub-class the OpenAI gym.Env module similar
to the way cartpole.py does.
Or do I need to create a whole new
PIP package as this article suggests?

Also, before I invest a lot of time on this, has anyone already created a cart-pole system environment where the goal is to stabilize the full state (not just the vertical position of the pendulum)?  (I tried googling and couldn't find any variants on the original cart-pole-v1 but I'm suspicious as I can't be the first person to make modified versions of some of these classic control environments).

Thanks, (I realize this question is a bit open-ended) but hoping for some good advice that will point me in the right direction.

This stackoverflow post provides a good answer.  It recommends creating the environment as a new package.

Note the updated link to the instructions on OpenAI.

So that's what I did and it's honestly not that much work and then you can import the environment into your script like this:

import gym
import gym_foo
env = gym.make('foo-v0')

This blog post also describes the process in a little bit more detail.


What are some limitations of using Collaborative Deep learning for Recommender systems? Recently I worked on a paper by Hao Wang, Collaborative Deep learning for Recommender Systems; which uses a two way tightly coupled method, Collaborative filtering for Item correlation and Stacked Denoising Autoencoders for the Optimization of the problem.

I want to know the limitations of using stacked Autoencoders and Hierarchical Bayesian methods to Recommender systems.

A good recommender program is one of the hardest problems in AI.
Recomender needs to first know me (by talking to me or by analyzing the movies that I graded as good).
Then it needs to be able to recognize similarity of my taste to a new movie. So it needs to watch movies that I liked, build a list of common things between these movies and then watch 100 random new movies and pick the one that has the most patterns in common with my liked movies.

So once again a good similarity recognition system is the key to successfull AI.


Is there any open source counterpart to the IBM Watson? I am looking for something similar to IBM Watson but open source.

I am not aware of any open-source products that can readily replace IBM Watson, but the following projects could be of interest to you in that regard:

DARPA DeepDive
Snorkel
Apache UIMA
YodaQA
OpenCog
OAQA (Open Advancement of Question Answering Systems)
Stanford CoreNLP – Natural language software
openQA


How can the sum of squared errors have negative gradient if it's defined as the squared of the error? The formula for the sum of squared errors (SSE) is:
$$
\frac{1}{2} \sum_{i=1}^n (t^i - o^i)^2
$$
I have a few related questions.

If $t^i - o^i$ is negative, doesn't the power of 2 eliminate any negative result?

How can then exist any negative gradient at the output?

What if the output is too high, and so the gradient should be negative, meaning that the weights (on average) have to be decreased? How can it be differentiated between 'output too high' and 'output too low'?

If $t^i - o^i$ is negative, doesn't the power of 2 eliminate any negative result?

In the loss function, yes that is correct, and is what you want - a measurement that gets higher due to any difference between the predicted and correct results. Minimising the value of that measurement is a goal for the optimiser.

How can then exist any negative gradient at the output?

The gradient of the cost function is different to the cost function. Technically it is the derivative of the cost function with respect to the parameters you can change. Taking the derivative changes the powers in the equation.
If you plot the graph from a single example, you can clearly see the gradient is negative when the prediction is too low, and positive when the prediction is too high:

Note this is the opposite relationship to the one you suggest in the question. That is why it is called gradient descent when we apply corrections. The steps to correct the parameters are in the opposite direction to the gradient.
In terms of algebra (and moving the example index into a subscript so that is doesn't get in the way visually), assuming $o_i$ is your output value, and $t_i$ is the target value, then you want the gradient of your loss function $L$ with respect to $o_i$ or $\frac{\partial L}{\partial o_i}$:
$$\frac{\partial L}{\partial o_i} =  \frac{\partial}{\partial o_i} \frac{1}{2}(t_i - o_i)^2 = \frac{\partial}{\partial o_i} \frac{1}{2}(t_i^2 - 2t_i o_i + o_i^2) = \frac{1}{2}(-2t_i + 2o_i) = o_i - t_i$$
This simple gradient value is negative if your output $o_i$ is lower than the target $t_i$, and positive if $o_i$is greater than $t_i$.
As an aside, if you wondered why the multiplier of $\frac{1}{2}$ was included in the definition of SSE, it is intended by design to end up with this simple result. Without it, everything works, just you would end up with the gradient being $2(o_i - t_i)$


How does Wolfram's Image Identification Project work? Wolfram Language Image Identification Project launched an Image Identify site demo which returns the top predicted tags for the photos.

How does it work, briefly? I mean what type of learning vision technologies are used to analyze, recognize and understand the content of an image?

The ImageIdentify project uses the highly automated "superfunctions" and as part of Wolfram Language API integration. It relies on a complex collection of meta-algorithms and built-in 'knowledge'. It has a built-in classifier trained from a large dataset using Wolfram Data Framework (WDF). However the main classifier is based on the deep neural networks.

Source: How the Wolfram Language Image Identification Project Works

The algorithm isn't perfect and misidentification are more likely to be caused by 'irrelevant objects repeatedly being in training images for a particular type of object'.

You can read more at Wolfram Language Artificial Intelligence: The Image Identification Project.


What is the difference between TensorFlow's callbacks and early stopping? What is the difference between TensorFlow's callbacks and early stopping?

Early stopping and callbacks are two different concepts:

Early stopping is a machine learning concept about when to stop training your model to avoid overfitting: You monitor a target value (e.g. validation loss) and stop learning after it hits a minimum. If the monitored value keeps increasing for a couple of epoch, you can restore the weights from the minimum.
Callbacks are a general software engineering concept and a technical implementation detail on how to trigger certain functions.

A callback function (e.g. early stopping) is a function that is passed as an argument to another function (e.g. fitting an ML model), and called in certain situations (e.g. after finishing an epoch). The callback function will perform its business (e.g. monitor the validation loss and decide whether to trigger the end of the training) and return control to the calling function (fitting the model).

In TensorFlow, early stopping is implemented using a callback function, but this is not the only way to do it. Also, there are further features in TensorFlow that are implemented using callbacks.


Multi-Label Image Classification using KNN I'm not quite sure how I should go about creating a multi-label image KNN classifier using python as a lot of the literature I have read does not explicitly explain this methodology. Specifically, I am not sure how I would be able to potentially yield multiple labels per image using the KNN classifier architecture. Any insight would be greatly appreciated! (new to coding by the way)

Have you tried google searching? There are many results that come up for "multi-label k nearest neighbor image classification". Here are the first couple of papers to get you started:

http://proceedings.mlr.press/v25/chiang12/chiang12.pdf
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.86.9501&rep=rep1&type=pdf


How to add contextual follow up like Google Assistant I am developing PDA like Google assistant on Android. So far, so good.
But now, I want to add contextual follow up like Google assistant so it can keep the train of thought.
As demonstrated here- https://www.youtube.com/watch?v=xYRENGuwwCA

Can anyone guide me or hint how to design the algorithm?
Thank you very much!

You would need to keep track of the current topic, and references. So, for example, a query When is the next train from London to Birmingham? would result in

topic = TRAIN_TRAVEL
start-loc = London
destination-loc = Birmingham

A follow-up And what about Bristol? would then replace destination-loc with "Bristol" and you would be able to build a new query from that.

The key issues here are the set of topics your assistant will be able to handle, and the relevant object slots. You might also want to clear the topic variable after the next input or so to avoid having it still hanging around even though the user no longer talks about that particular topic.

UPDATE: Just to add how that would work from a technical point of view. The input And what about Bristol? would not be recognised as any known intent, as it is too unspecific. As a fallback your Assistant then takes the last topic, and interprets the input in the light of that context. So here we assume that the intent is 'train travel', as that is the last thing the user spoke about. This will of course not always work, but should sort out the majority of cases.


Is it possible to have the latent vector of an auto-encoder with size 1? Given e.g. 1M vectors of $1000$ floating points each, where every point in vectors is sampled from a uniform distribution between $-1$ to $1$:

Is it possible to have the bottleneck of the AE network with size 1? In other words, without caring about generalization, is it possible to train a network, where, given only 1 encoded value, it can recreate any of the 1M examples?

According to various experimentation on autoencoders, it is very possible to have latent vector of size 1. Various layers can help the downsizing of the original input to a very small size of 1. But an issue may arise during decoding. If you're expecting that through one or two or maybe five layers in decoder you can achieve an accurate reconstruction, it is highly unlikely abd the result will turn out to be blurry. Maybe a great network with various parameters may help the reconstruction without considering generalization as asked by you.


Is it realistic to train a transformer-based model (e.g. GPT) in a self-supervised way directly on the Mel spectrogram? In music information retrieval, one usually converts an audio signal into some kind "sequence of frequency-vectors", such as STFT or Mel-spectrogram.
I'm wondering if it is a good idea to use the transformer architecture in a self-supervised manner -- such as auto-regressive models, or BERT in NLP -- to obtain a "smarter" representation of the music than the spectrogram itself. Such smart pretrained representation could be used for further downstream tasks.
From my quick google search, I found several papers which do something similar, but -- to my surprise -- all use some kind of symbolic/discrete music representation such as scores. (For instance here or here).
My question is this:

Is it realistic to train such an unsupervised model directly on the
Mel spectrogram?

The loss function would not be "log softmax of next word probability", but some kind of l2-distance between "predicted vector of spectra" and "observed vector of spectra", in the next time step.
Did someone try it?

The reason most music-generation models use discrete representations is because the long-term structures of music are very challenging to model. Note that MAESTRO (the dataset behind the two papers you linked) contains performances, not scores, so they include timing and accents of real performers--but are still sequences of discrete events, not audio.
There's been some work on learning discrete representations directly from audio, such as with vector-quantized variational autoencoders (VQ-VAEs). Typically an autoregressive model is trained on top of the learned representation; Jukebox used a transformer for that. By the way, I'd highly recommend reading through the "related work" section of the Jukebox paper for an overview of work on the audio/speech synthesis task.
wav2vec is probably closest to what you're describing. They train a transformer on raw audio, self-supervised, in order to learn good representations of human speech for the speech-to-text task.
As far as training directly on spectograms, there's MelNet, a somewhat exotic RNN trained for a variety of audio synthesis tasks, including music.
Hope this helps!


How to convert sequences of images into state in DQN? I recently read the DQN paper titled: Playing Atari with Deep Reinforcement Learning. My basic and rough understanding of the paper is as follows:

You have two neural networks; one stays frozen for a duration of time steps and is used in the computation of the loss function with the neural network that is updating. The loss function is used to update the neural network using gradient descent.

Experience replay is used, which basically creates a buffer of experiences. This buffer of experiences is randomly sampled and these random samples are used to update the non-frozen neural network.

My question pertains to the DQN algorithm illustrated in the paper: Algorithm 1, more specifically lines 4 and 9 of this algorithm. My understanding, which is also mentioned early on in the paper, is that the states are actually sequences of the game-play frames. I want to know, since the input is given to a CNN, how would we encode these frames to serve as input to the CNN?

I also want to know since $s_{1}$ is equal to a set, which can be seen in line 4 of the algorithm, then why is $s_{t+1}$ equal to $s_{t}$, $a_{t}$, $x_{t+1}$?

I want to know, since the input is given to a CNN, how would we encode these frames to serve as input to the CNN?

As nbro mentioned in a comment to your answer, this question has very recently been asked and answered here.

  I also want to know since $s_1$ is equal to a set, which can be seen in line 4 of the algorithm, then why is $s_{t+1}$ equal to $s_t$, $a_t$, $x_{t+1}$?

The algorithm presented in the original DQN paper is relatively simple and written to express the main ideas of their approach (e.g. experience replay, preprocessing histories, gradient descent, etc.); in fact, it isn't even the exact algorithm used in the experiments! For example, the experiments use frame-skipping to reduce computation - this is not mentioned in Algorithm 1 in the paper. With that in mind, setting $s_{t+1}$ equal to $s_t, a_t, x_{t+1}$ in the algorithm signifies a general notion of constructing the next raw state $s_{t+1}$ from the previous preprocessed state $s_t$, previous action $a_t$, and current frame $x_{t+1}$. For example:

If the action space at the next timestep is constrained by the state, then the state may need additional parameters to encode the action space.
The algorithm needs some indication if a state is terminal, and such an indication may need to be encoded in the state.
If there is frame skipping, then multiple frames will be needed to construct the next state, possibly using the previous state as well.

The above examples should display how the encoding of the state cannot always simply be a stack of raw frames, or even a function of $s_t$, $a_t$ and $x_{t+1}$, and therefore a more general approach is often required.


Can we create AI to not only recognize itself, but to recognize other AI systems as well? Can AI systems be created that could recognize itself, and recognize intelligence in other systems, and make intelligent decisions about the other systems? Mankind seems to be making progress in self-recognition but I've not seen evidence of one system recognizing other systems and being able to compare it's own intelligence with other systems. How could this be accomplished?

In the abstract, mechanisms for self-recognition (I personally prefer the phrase 'metacognition', since it carries fewer spurious associations) and recognition of intelligence in other systems can be considered to be pretty much equivalent.

In fact, both can be characterized in the standard percept/action framework: the task in both cases involves (however coarsely) classifying/predicting the behaviour of a black box system.

Such tasks can be universally characterized in terms of frameworks such as Solomonoff Induction or (more recently) AIXI.


Can we get the inverse of the function that a neural network represents? I was wondering if it's possible to get the inverse of a neural network. If we view a NN as a function, can we obtain its inverse?

I tried to build a simple MNIST architecture, with the input of (784,) and output of (10,), train it to reach good accuracy, and then inverse the predicted value to try and get back the input - but the results were nowhere near what I started with. (I used the pseudo-inverse for the W matrix.)

My NN is basically the following function:

$$
f(x) = \theta(xW + b), \;\;\;\;\; \theta(z) = \frac{1}{1+e^{-z}}
$$

I.e.

def rev_sigmoid(y):
    return np.log(y/(1-y))

def rev_linear(z, W, b):
    return (z - b) @ np.linalg.pinv(W)

y = model.predict(x_train[0:1])
z = rev_sigmoid(y)
x = rev_linear(z, W, b)
x = x.reshape(28, 28)
plt.imshow(x)

^ This should have been a 5:

Is there a reason why it failed? And is it ever possible to get inverse of NN's?

EDIT: it is also worth noting that doing the opposite does yield good results. I.e. starting with the y's (a 1-hot encoding of the digits) and using it to predict the image (an array of 784 bytes) using the same architecture: input (10,) and output (784,) with a sigmoid. This is not exactly equivalent, to an inverse as here you first do the linear transformation and then the non-linear. While in an inverse you would first do (well, undo) the non-linear, and then do (undo) the linear. I.e. the claim that the 784x10 matrix is collapsing too much information seems a bit odd to me, as there does exist a 10x784 matrix that can reproduce enough of that information.

Mathematical Exploration

let $\Theta^+$ be the pseudo-inverse of $\Theta$.

Recall, that if a vector $\boldsymbol v \in R(\Theta)$ (ie in the row space) then $\boldsymbol v = \Theta^+\Theta\boldsymbol v$. That is, so long as we select a vector that is in the rowspace of $\Theta$ then we can reconstruct it with full fidelity using the pseudo inverse. Thus, if any of the images happen to be linear combinations of the rows of $\Theta$ then we can reconstruct it.

To be more specific. let $f(\boldsymbol x)$ have a pseudo-inverse $f^+(\boldsymbol x)$ defined as you have. If we restrict our domain such that $\boldsymbol x \in C(\Theta^T)$ (column space of the transpose) then $f^+= f^{-1}_{res}$.

That is, under our domain restriction the pseudo inverse becomes a true inverse.

An Extrapolation

It would then seem that so long as we are under such domain restrictions then we could define a pseudo inverse for a general NN. Though, it might be possible that some NNs don't have any restriction that admits an inverse. Maybe, there is some way to regularize the parameters such that this is possible. NNs with ReLU wouldn't admit such an inverse since ReLU loses information on negative values. Leaky Relu might work.

Further Investigation

Finally, this presents a zone for further study. Some questions to answer might be:

Is it possible for optimized parameters to contain non-trivial examples in their row-space?
If so, under what conditions is this possible?
Are the examples in any way represented in the row space?
Is there some way to regularize a NN such that it admits an inverse over some desired restriction?
Under what conditions is invertibility useful?


How to store datasets of lexical connections? I'm investigating the possibility of storing the semantic-lexical connections (such as the relationships to the other words such as phrases and other dependencies, its strength, part of speech, language, etc.) in order to provide analysis of the input text.

I assume this has been already done. If so, to avoid reinventing the wheel, is there any efficient method to store and manage such data in some common format which has been already researched and tested?

I would think you could use a graph database, perhaps Neo4J or Titan or something of that nature.  Or, if you want a simple file format, you could use one of the many formats that exist for representing graphs.  You can find a list and overview of some of them here.

Another option would be to store them in RDF using a triplestore like Jena.


Measuring novel configuration of points I am trying to implement Novelty search; I understand why it can work better than the standard Genetic Algorithm based solution which just rewards according to the objective.
I am working on a problem which requires to generate a fixed number of points in a 2d box centered at the origin.
In this problem, how can I identify which is a novel configuration of points?
Note: I have thought of one way of doing this: We call the mean of one configuration of points to be the mean of all points in that configuration (let's say this tuple is $(m_x, m_y)$, we store the mean of all configurations generated till now, now for a new configuration it's novelty can be defined as the distance of the mean of this new configuration with $(m_x, m_y)$.
But I think it will not work greatly as some very different configuration of points can also have the same mean.

You can define different measures in this way:

Maximum distance of the new point with all points of the configuration ($M$)
Minimum distance of the new point with all points of the configuration ($N$)
$\frac{M}{\text{Maximum distance between two points of the configuration(}D)}$: normalize (1) measure
$\frac{N}{D}$:normalize (2) measure

You can get more ideas from distance measures in hierarchical clustering methods. To select a proper one, you need to elaborate on the context of these points.


Has there been "prescriptive" attempts at language-learning/comprehension in AI? Is there examples of AI systems being taught grammatical rules, or have such rules built-in to them, in order to recognize commands and the like in natural language?

For example, is there a program, when given the instruction (either by speech or by text) in the form of "Open file-1", will attempt to understand the command as consisting the parts "open" and "file-1", then respond according to how each part is classified?

The focus here is the fact that these kind of instructions are processed with built-in rules of grammar, rather than the rules being learned through observation of data.

Yes.  You're basically looking for "rule based" NLP.  For reference, see:

https://en.wikipedia.org/wiki/Rule-based_system

https://www.quora.com/Why-are-rule-based-methods-becoming-unpopular-in-NLP-Are-rule-based-methods-still-in-use-If-yes-where-should-I-look-for-them

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3756254/

https://nlp.stanford.edu/software/regexner.html

http://u.cs.biu.ac.il/~yogo/courses/sem2014/intro.pdf

etc.


How can the $\lambda$-return be defined recursively? The $\lambda$-return is defined as
$$G_t^\lambda = (1-\lambda)\sum_{n=1}^\infty \lambda^{n-1}G_{t:t+n}$$
where
$$G_{t:t+n} = R_{t+1}+\gamma R_{t+2}+\dots +\gamma^{n-1}R_{t+n} + \gamma^n\hat{v}(S_{t+n})$$
is the $n$-step return from time $t$.

How can we use this definition to rewrite $G_t^\lambda$ recursively?

To rewrite $G_t^\lambda$ recursively, our goal is to define it in terms of
$$G_{t+1}^\lambda = (1-\lambda)\sum_{n=1}^\infty \lambda^{n-1}G_{t+1:t+n+1}.\tag{0}$$

The $\lambda$-return is a weighted average of all $n$-step returns. We will split up the summation by  pulling out the one-step return $G_{t:t+1}$ and the first step's reward $R_{t+1}$.

$$
\begin{align*}
G_t^\lambda &= (1-\lambda)\sum_{n=1}^\infty \lambda^{n-1}G_{t:t+n} \tag{1}\\
&\\
&= (1-\lambda)\lambda^0G_{t:t+1} + (1-\lambda)\sum_{n=2}^\infty \lambda^{n-1}G_{t:t+n}\tag{2}\\
&\\
&= (1-\lambda)\left(R_{t+1}+\gamma\hat{v}(S_{t+1})\right)\\
&\qquad + (1-\lambda)\sum_{n=2}^\infty \lambda^{n-1}(R_{t+1}+\gamma R_{t+2}+\dots +\gamma^{n-1}R_{t+n} + \gamma^n\hat{v}(S_{t+n}))\tag{3}\\
&\\
&= (1-\lambda)\left(R_{t+1}+\gamma\hat{v}(S_{t+1})\right) + (1-\lambda)\sum_{n=2}^\infty \lambda^{n-1} R_{t+1}\\
&\qquad + (1-\lambda)\sum_{n=2}^\infty \lambda^{n-1}(\gamma R_{t+2}+\dots +\gamma^{n-1}R_{t+n} + \gamma^n\hat{v}(S_{t+n}))\tag{4}\\
&\\
&= \gamma(1-\lambda)\hat{v}(S_{t+1}) + (1-\lambda)\sum_{n=1}^\infty \lambda^{n-1} R_{t+1}\\
&\qquad + (1-\lambda)\sum_{n=2}^\infty \lambda^{n-1}(\gamma R_{t+2}+\dots +\gamma^{n-1}R_{t+n} + \gamma^n\hat{v}(S_{t+n}))\tag{5}\\
&\\
&= \gamma(1-\lambda)\hat{v}(S_{t+1}) + R_{t+1}\\
&\qquad + (1-\lambda)\sum_{n=2}^\infty \lambda^{n-1}(\gamma R_{t+2}+\dots +\gamma^{n-1}R_{t+n} + \gamma^n\hat{v}(S_{t+n}))\tag{6}\\
&\\
&= \gamma(1-\lambda)\hat{v}(S_{t+1}) + R_{t+1}\\
&\qquad + \gamma\lambda(1-\lambda)\sum_{n=2}^\infty \lambda^{n-2}(R_{t+2}+\dots +\gamma^{n-2}R_{t+n} + \gamma^{n-1}\hat{v}(S_{t+n}))\tag{7}\\
&\\
&= \gamma(1-\lambda)\hat{v}(S_{t+1}) + R_{t+1}\\
&\qquad + \gamma\lambda(1-\lambda)\sum_{m=1}^\infty \lambda^{m-1}(R_{t+2}+\dots +\gamma^{m-1}R_{t+m+1} + \gamma^{m}\hat{v}(S_{t+m+1}))\tag{8}\\
&\\
&= \gamma(1-\lambda)\hat{v}(S_{t+1}) + R_{t+1} + \gamma\lambda(1-\lambda)\sum_{m=1}^\infty \lambda^{m-1}G_{t+1:t+m+1}\tag{9}\\
&\\
&= \gamma(1-\lambda)\hat{v}(S_{t+1}) + R_{t+1} + \gamma\lambda G_{t+1}^\lambda \tag{10}\\
\end{align*}
$$

$ $
$(2)$ pulls out the one-step return from the summation.
$(3)$ expands the $n$-step returns.
$(4)$ pulls out the remaining first step rewards.
$(5)$ combines first step rewards.
$(6)$ simplifies the geometric series.
$(7)$ pulls a factor of $\gamma\lambda$ out of the summation.
$(8)$ makes the substitution $m=n-1$.
$(9)$ uses the definition of the $n$-step return.
$(10)$ uses the definition of the $\lambda$-return

The result can be verified in equation $(12.18)$ of Sutton and Barto's RL book.


What is the smoothness assumption in SVMs? In this research paper, we have the following claim

the smoothness assumption that underlies many kernel methods such as Support Vector Machines (SVMs) does not hold for deep neural networks trained through backpropagation

Does smoothness here refer to no sharp rise/fall in gradients?

Smoothness here is the mathematical definition, so as you implied smoothness is ruled out by output data with sharp spikes or discontinuous jumps (and possibly the data of the gradient, the gradient's gradient, ad infinitum, depending on who defines smoothness).
By any definition a lot of activation functions are not smooth, for example RELU. This means neural networks in general are not smooth.


Library for rendering neural network NEAT I just finished my implementation of NEAT and I want to see the phenotype of each genome. Is there a library for displaying a neural network like this?

Example of my genome syntax:
[[0, 11, 0.9154901559275923, 1, 19],
[4, 11, 1.3524964932656411, 1, 19],
[12, 9, -1.755210214894685, 1, 23],
[11, 12, 0.6193383549414015, 1, 23]]

Where [In, Out, Weight, Activated?, Innovation]

I've never used it, but, if you are using Python, have a look at neat-python, which is a Python package that implements NEAT and also provides a module, visualize, to plot the best and average fitness vs. generation, plot the change in species vs. generation, and to show the structure of a network described by a genome. See this Stack Overflow thread, if you encounter any issue while attempting to use this module. See also this example.

Have also a look at this web page, which lists links to many NEAT implementations in different languages, where some of them provide visualization tools.


Which "assumptions" made about the state space are Russell and Norvig referring to in their book? I am reading the cornerstone book, "Artificial Intelligence, A Modern Approach" by Stuart Russel, and Peter Norvig, and there is a passage in the book on page 98:

The complexity results depend very strongly on the assumptions made
about the state space. The simplest model studied is a state space
that has a single goal and is essentially a tree with reversible
actions. (The 8-puzzle satisfies the first and third of these
assumptions.)

What are the "assumptions" in that context?

The complexity results depend very strongly on the assumptions made about the state space. The simplest model studied is a state space that has a single goal and is essentially a tree with reversible actions.

The assumptions are the following:

  The simplest model studied is a state space

  that has a single goal

  and is essentially a tree with reversible actions.


What is the state-space complexity of Spades? AI reached super-human level in many complex games, including imperfect information games such as six-player no-limit Texas hold’em poker.  However, it still did not reached that level in Trick-taking card games such as Spades, Bridge, Skat and Whist.  In a related question, I am asking Why Trick-Taking games are a challenge for AI.
An important factor that makes those games a challenge for AI is their size, to be precise lets talk about the State-space complexity which is define as the number of legal game positions reachable from the initial position of the game  [Victor Allis (section 6.2.4)].
What is the size of Spades?

I have calculated an upper bound and modified a calculation of a lower bound for a similar game.  I assume the real size is closer to the upper bound.
$$ 7.36 \cdot 10^{27} \leq |SPADES| \leq 3.09 \cdot 10^{72} $$
Upper bound
The State-space complexity of Spades can be computed by counting for each possible starting state (deal), all the possible sequences of actions.  While that calculation could be hard, an upper bound is easy to achieve by multiplying the number of initial positions $|S_0|$ by an upper bound on the number of different possible histories $|\bar{H}|$.
$$ |SPADES| = \sum_{s \in S_0} |H_{s}| \leq |S_0| \cdot |\bar{H}| $$
The number of possible starting positions $|S_0|$ is the number of different ways to deal a 52 cards deck into 4 players
$$|S_0| = \frac{52!}{13!^4} \approx 5.3\cdot10^{28}$$
An information set in Spades contains two objects: the player's hand, and the sequence of actions each of the players have made from the start of the round.  Before any card is played, each information set (a single hand) can be completed into a full game-state (4 hands) in the following number of ways
$$ |\bar{S}| = \frac{39!}{13!^3} \approx 8.45 \cdot 10^{16} $$
During a round, each player first decide upon a bid, then choose one out of her 13 cards, than choose one out of 12, etc. Thus, the size of the decision tree, which is the number of histories is bounded by
$$|H| \leq {14!}^4 \approx 5.77 \cdot 10^{43}$$
This is an upper bound since some of those histories violates the rules of the game.
Therefore, the number of legal positions in Spades is bounded by
$$ |SPADES| = \sum_{s \in S_0} |H_{s}| \leq |S_0| \cdot |H| \leq \frac{52!}{13!^4} \cdot {14!}^4  \approx 3.09 \cdot 10^{72} $$
Lower bound
In the paper: Understanding the Success of Perfect Information Monte Carlo Sampling in Game Tree Search, Sturtevant et al. describe a loose lower bound for the size of Skat:

Skat is a 3-player card game with 32 cards, from which 10 are dealt
to each player. There are $$ {32 \choose 10} = 364, 512, 240 $$hands each
player can have and
$$ H := {22 \choose10} \cdot {12\choose10} = 42, 678, 636 $$
hands for the other players which is what constitutes an information set at the start of a game. At the beginning of each trick, the trick leader can choose to play any of his remaining cards. Therefore, there are at least
$$ 10!H \approx 1.54 · 10^{14}$$
information sets. But, from an opponents perspective there
are actually 20 or 22 unknown cards that can be lead, so this
is only a loose lower bound on the size of the tree. This
should clearly establish that even when using short decks it
is infeasible to solve even a single game instance of a trick-based
card game.

Using the same logic to produce a loose lower bound on the size of Spades tree produce the following:
$$ H := {39 \choose 13} \cdot {26 \choose 13} = 8.45 \cdot 10^{16} $$
At the start of the round (before bidding has been made) the number of information sets is lower bounded by
$$ 14!H \approx  7.36 \cdot 10^{27}$$


Why do ResNets avoid the vanishing gradient problem? I read that, if we use the sigmoid or hyperbolic tangent activation functions in deep neural networks, we can have some problems with the vanishing of the gradient, and this is visible by the shapes of the derivative of these functions. ReLU solves this problem thanks to its derivative, even if there may be some dead units. ResNet uses ReLU as activation function, but looking online what I understood is that ResNet solves the vanishing of the gradient thanks to its identity map, and I do not totally agree with that. So what's the purpose of the identity connections in ResNet? Are they used for solving the vanishing of the gradient? And ReLU really solves the vanishing of the gradient in deep neural networks?

Before proceeding, it's important to note that ResNets, as pointed out here, were not introduced to specifically solve the VGP, but to improve learning in general. In fact, the authors of ResNet, in the original paper, noticed that neural networks without residual connections don't learn as well as ResNets, although they are using batch normalization, which, in theory, ensures that gradients should not vanish (section 4.1). So, in this answer, I'm just giving a potential explanation of why ResNets may also mitigate (or prevent to some extent) the VGP, but the cited research papers below also confirm that ResNets prevent the VGP. Given that I didn't fully read all the papers mentioned in this answer, the information in this answer may not be fully accurate.
The skip connections allow information to skip layers, so, in the forward pass, information from layer $l$ can directly be fed into layer $l+t$ (i.e. the activations of layer $l$ are added to the activations of layer $l+t$), for $t \geq 2$, and, during the forward pass, the gradients can also flow unchanged from layer $l+t$ to layer $l$.
How exactly could this prevent the vanishing gradient problem (VGP)? The VGP occurs when the elements of the gradient (the partial derivatives with respect to the parameters of the NN) become exponentially small, so that the update of the parameters with the gradient becomes almost insignificant (i.e. if you add a very small number $0 < \epsilon \ll 1$ to another number $d$, $d+\epsilon$ is almost the same as $d$) and, consequently, the NN learns very slowly or not at all (considering also numerical errors). Given that these partial derivatives are computed with the chain rule, this can easily occur, because you keep on multiplying small (finite-precision) numbers (please, have a look at how the chain rule works, if you're not familiar with it). For example, $\frac{1}{5}\frac{1}{5} = \frac{1}{25}$ and then $\frac{1}{5}\frac{1}{25} = \frac{1}{125}$, and so on. The deeper the NN, the more likely the VGP can occur. This should be quite intuitive if you are familiar with the chain rule and the back-propagation algorithm (i.e. the chain rule). By allowing information to skip layers, layer $l+t$ receives information from both layer $l+t-1$ and layer $l$ (unchanged, i.e. you do not perform multiplications). For example, to compute the activation of layer $l+t-1$, you perform the usual linear combination followed by the application of the non-linear activation function (e.g. ReLU). In this linear combination, you perform multiplications between numbers that could already be quite small, so the results of these multiplications are even smaller numbers. If you use saturating activation functions (e.g. tanh), this problem can even be aggravated. If the activation of layer $l+t$ are even smaller than the activations of layer $l+t-1$, the addition of the information from layer $l$ will make these activations bigger, thus, to some extent, they will prevent these activations from becoming exponentially small. A similar thing can be said for the back-propagation of the gradient.
Therefore, skip connections can mitigate the VGP, and so they can be used to train deeper NNs.
These explanations are roughly consistent with the findings reported in the paper Residual Networks Behave Like Ensembles of Relatively Shallow Networks, which states

Our results reveal one of the key characteristics that seem to enable the training of very deep networks: Residual networks avoid the vanishing gradient problem by introducing short paths which can carry gradient throughout the extent of very deep networks.

In the paper Norm-Preservation: Why Residual Networks Can Become Extremely Deep?, the authors also discuss another desirable effect of skip connections.

We show theoretically and empirically that each residual block in ResNets is increasingly norm-preserving, as the network becomes deeper


Can singularity result into the extinction of the third world? Based on Darwin's statement, "it is not the strongest that survives; but the species that survives is the one that is able to adapt to and to adjust best to the changing environment". Can economical constraints(not being able to afford for researches and developments) or religious beliefs (such as the belief of nothing can outperform the creations of god) prevent third world countries from catching up to these evolutionary progresses? if they couldn't, would it result into the extinction of their societies?

Since we're in the Artificial Intelligence StackExchange, I'll approach the answer from that perspective. The singularity (in this case superintelligent artificial general intelligence (AGI)) tends to have global consequences. In Superintelligence: Paths, Dangers, Strategies Bostrom elaborates on this idea. Essentially, there are three potential outcomes (keep in mind these are from a human perspective.):

Global positive
Global negative
Uneven outcomes

Global positive assumes the AGI has goals which align with human values (likely because we sorted out the control problem/alignment). The outcome here ranges between humans simply continuing to exist (maybe the AGI simply leaves) and humans receiving a cosmic endowment (utopian singularity stuff).

Global negative assumes the AGI does not have goals which align with human values (likely because we didn't sort out the control problem/alignment).The outcome here ranges from humans simply continuing to exist (again, the AGI may simply leave), to humans no longer existing (perhaps we are more useful to the unaligned AGI as paper clips).

The global outcomes appear to be significantly more likely due to the way we are approaching the control problem/alignment. Aligning an AGI with human goals in general is extremely difficult. Aligning an AGI with specific human's goals implies total control over it, which is significantly more difficult. With that said, it may happen, which could lead to the third potential outcome.

Uneven outcomes assume the control problem has been completely solved, and the AGI is perfectly aligned with a specific group of human's goals (to reiterate, this is incredibly difficult to do, and is unlikely to happen.) In this scenario, the controlling group would have a decisive strategic advantage, receive a cosmic endowment, and would likely form a singleton. Perhaps they would share with others, perhaps they would turn everybody else into paper clips, who knows. On this path, groups which do not pursue AGI (perhaps for the reasons you listed) are taking an existential risk.

So, in general, it is unlikely things will go particularly well for one group while going badly for another, but if it does it may be quite bad for the losing groups.


Constraining the output value range of a CNN independent of the loss function I'm having the following problem: `

I'm training a multi-output CNN and using the relative values of the outputs in my loss function. The net is learning well, but as the absolute values of the outputs are not regularized in anyway in the loss function, the values of the outputs keep rising. This causes a situation where the result I need (values of the outputs relative to each other) are quite right, but the huge absolute values produce underflow resulting NaN values at some point of the training. Is there some way to constrain these absolute values (perhaps elsewhere than the loss function?)

I'm using a custom loss function implementing a recovery angular error metric, so the loss function is somewhat complex. A weighed mean of the net outputs is fed to this function, so using both the net outputs and the weighed mean would lead to some quite problematic gradient derivation. Would it be maybe possible to constrain the value range of the net outputs in the net structure itself?

You can limit the absolute values of the outputs by "punishing" large values in the loss function. This is done by adding an extra term to the loss function.

For example, if your existing loss function is L(yhat, y) where yhat is the output and y the correct value, create a new loss function

L'(yhat, y) = L(yhat,y) + max(0, ||yhat||1 - k)

When the outputs are "small" the L1 norm of your output yhat is less than k and imposes no extra loss (keeping everything as before). However once the norm grows above k it starts to impose a loss that guides the training to reduce the absolute values of the outputs values.

You can also limit the values in the network itself by selecting a bounded activation function for the output, for example tanh or sigmoid. However, you need to pick an activation function that suits your problem.


Is there a way of pre-determining whether a CNN model will perform better than another? I developed a CNN for image analysis. I've around 100K labeled images.  I'm getting a accuracy around 85% and a validation accuracy around 82%, so it looks like the model generalize better than fitting. So, I'm playing with different hyper-parameters: number of filters, number of layers, number of neurons in the dense layers, etc.

For every test, I'm using all the training data, and it is very slow and time consuming.

Is there a way to have an early idea about if a model will perform better than another?

I would like you to try with following changes in the model.

Introduce batch normalization layer in the model.
Try with the batch size of 32 - 64
Use different architecture, Like VGG, Resnet, etc..

There is no full proof answer to this question but you can get best by trying some know strategies.


What does it mean to parameterise a policy in policy gradient methods? Can you explain policy gradient methods and what it means for the policy to be parameterised? I am reading Sutton and Barto book on reinforcement learning and didn't understand well what it is, can you give some examples?

In the context of RL, for a policy to be parameterised it typically means we explicitly model the policy and is common in policy gradient methods.

Consider value based methods such as Q-learning where our policy is usually something like $\epsilon$-greedy where we choose our action using the following policy

\begin{align}
\pi(a|s) = \left\{ \begin{array}{ll}
\arg \max_a Q(s,a) & \text{with probability } 1-\epsilon\;; \\ \text{random action} & \text{with probability } \epsilon\;.
\end{array}\right.
\end{align}
Here we have parameterised the policy with $\epsilon$ but the learning is done by learning the Q-functions. When we parameterise a policy we will explicitly model $\pi$ by the following:
$$\pi(s|a,\boldsymbol{\theta}) = \mathbb{P}(A_t = a | S_t=s, \boldsymbol{\theta}_t = \boldsymbol{\theta})\;.$$
Learning is now done by learning the parameter $\boldsymbol{\theta}$ that maximise some performance measure $J(\boldsymbol{\theta})$ by doing approximate gradient ascent updates of the form
$$\boldsymbol{\theta}_{t+1} = \boldsymbol{\theta}_t + \alpha \hat{\Delta J(\boldsymbol{\theta}_t)}. $$

Note that, as per the Sutton and Barto textbook, $\hat{\Delta J(\boldsymbol{\theta}_t)}$ is a noisy, stochastic estimate of $\Delta J(\boldsymbol{\theta}_t)$ where the former approximates the latter in expectation.

The policy can be parameterised in any way as long as it is differentiable with respect to the parameters. Commonly in Deep RL the policy is parameterised as a neural network so $\boldsymbol{\theta}$ would be the weights of the network.


What does it mean for AlphaZero's network to be "fully trained" Reading this blog post about AlphaZero:
https://deepmind.com/blog/article/alphazero-shedding-new-light-grand-games-chess-shogi-and-go

It uses language such as "the amount of training the network needs" and "fully trained" to describe how long they had the machine play against itself before they stopped training. They state training times such as 9 hours, 12 hours, and thirteen days for chess, shogi, and Go respectively. Why is there a point at which the training "completes?" They show plots of AlphaZero's performance on the Y axis (its Elo rating) as a function of the number of training steps. Indeed, the performance seems to level out as the number of training steps increases beyond a certain point. Here's a picture from that site of the chess performance vs training steps:

Notice how sharply the Elo rating levels off as a function of training steps.

First: am I interpreting this correctly? That is, is there an asymptotic limit to improvement on performance as training sessions tend to infinity?
If I am interpreting this correctly, why is there a limit? Wouldn't more training mean better refinement and improvement upon its play? It makes sense to me that the millionth training step may yield less improvement than the very first one, but I wouldn't expect an asymptotic limit. That is, maybe it gets to about 3500 Elo points in the first 200k training steps over the course of the first 10 hours or so of playing ches. If it continued running for the rest of the year, I'd expect it to rise significantly above that. Maybe double its Elo rating? Is that intuition wrong? If so, what are the factors that limit its training progress beyond the first 10 hours of play?

Thanks!

Neural network will eventually reach limit of it's approximation power. You can't expect to learn more and more things infinitely long with the same amount of learnable parameters. Also, if you eventually reach optimal performance, you can't play more optimal than what optimal is (not saying that it reached optimal performance but possibly something close to optimal for it approximation abilities). So probably combination of those two leads for the performance increase to reach it's limit.


How should I deal with variable input sizes for a neural network classifier? I am currently working on a project, where I have a sensor in a shoe that records the $X, Y, Z$ axes, from an acceleration and gyroscope sensor. Every millisecond, I get 6 data points. Now, the goal is, if I do an action, such a jumping or kicking, I would use the sensor's output to predict that action being done.

The issue: If I jump, for example, one time I may get 1000 data points, but, in another, I get 1200 amounts, meaning the size of the input is different.

The neural networks I've studied so far require the input size to be constant to predict a $Y$ value, however, in this case, it isn't. I've done some research on how to make a neural network with variable sizes, but haven't been able to find one which works. It's not a good idea to crop the input to a certain size, because then I am losing data. In addition, if I just resize the smaller trials by putting extra $0$s, it skews the model.

Any suggestions on a model that would work or how to better clean the data?

It is much simpler to process the data in a different way. Since you're using temporal data a common practice is to define a priori a minimum time-step, usually called $\textit{granularity}$, which must be bigger than you're sensor responsiveness. Using this granularity value you'll then be able split your data into intervals, and you can then combine each instance belonging to an interval with a function that you like. Most common choice is obviously averaging the values, but also summing then could be a choice or a moving average.

Don't think that in this way you will loose information, preprocessing is also called data cleaning for a reason, not aways raw data are better.

As a last note, I would suggest you to look into 'Machine learning for the quantified self', there is growing literature going on about using sensors to train predictive models about movements and body measures like heart-rate. You might find something about preprocessing for this specific sensor-applications.


Which NN Arch? - Given input vec X -> Output vector Y made of "not binary", floating point values I am thinking of an application where I want to teach a Neural Network to adjust some parameters on another device for a specific purpose.
I say "adjust some parameters" because I need the output to "turn the knobs" of something else, not only binary "push buttons"

What kind of neural network do I need? is it a specific architecture?

The NNs I have used so far were sequential and outputting vectors like
[0] or [1]
and classifiers with outputs like [1,0,0,0] [0,1,0,0] [0,0,1,0] [0,0,0,1] for example

But now I need outputs like [0.25, 0.99, 0.14, 0.54],

0.25 for knob 1
0.99 for knob 2
0.14 for knob 3
0.54 for knob 4

Thank you for your answers

This is a regression problem instead of a classification problem. For both, you can use a feedforward neural network. These are the typical ones you've probably seen where each neuron in a layer is connected to each neuron in the next.

In a classification problem, you want an output like you mentioned above, a one-hot vector corresponding to the correct class. The way you get this vector is by applying a soft-max activation to the outputs of your last layers. If you don't apply this activation, your output vector might be something like [-2.5, 0.6, 10.2, 5.1]. You can see that this looks a lot like the vector you want. So, you can probably use a feedforward architecture (with non-linear activations on the hidden layers if there are non-linearities in your data) with a final layer of 4 neurons. Then, when you train, that should (hopefully) converge to the correct data outputs. If it doesn't, you will need to mess with the architecture/get more training data/lots of other levers to play with.

This link goes to a tutorial for regression using Keras. Keras is a nice package written to take powerful libraries like TensorFlow and Theano and give them a simple interface. TensorFlow can be quite complex (I've never used Theano so I don't know) when you're starting and trying to understand the syntax and the ML concepts. Keras helps take away the confusing syntax.


Is it possible to modify or replace the basic network of YOLO? I have an idea to adapt YOLO algorithm to my application, the original YOLO algorithm is for image classifications, which have 24 convolutional layers with output class of 1000, is it possible to replace the basic network of YOLO with Alexnet or Resnet or the custom network structure designed by myself? Noted that my application have input shape of 500 * 10000 * 1 and only 4 classes for classification.

No, the original (or any) YOLO is for object detection. You can easily replace the feature extractor (DarkNet53, if I'm not mistaken) with any other, as long as you maintain the correct number of weights in the detection layer.


What are the conceptual differences between regularisation and optimisation in deep neural nets? I'm trying to wrap my mind around the concepts of regularisation and optimisation in neural nets, especially around their differences.
In my current understanding, regularisation is intended to tackle overfitting whereas optimisation is about convergence.
However, even though regularisation adds terms to the loss function, both approaches seem to do most of their things during the update phase, i.e. they work directly on how weights are updated.
If both concepts are focused on updating weights,

what are the conceptual differences, or why aren't both L2 and Adam, for example, called either optimisers or regularisers?

Can/should I use them together?

You are correct.
The main conceptual difference is that optimization is about finding the set of parameters/weights that maximizes/minimizes some objective function (which can also include a regularization term), while regularization is about limiting the values that your parameters can take during the optimization/learning/training, so optimization with regularisation (especially, with $L_1$ and $L_2$ regularization) can be thought of as constrained optimization, but, in some cases, such as dropout, it can also be thought of as a way of introducing noise in the training process.
You should use regularisation when your neural network is big (where big, of course, is not well-defined) and you have little data (where little is also not well-defined). Why do you want to use regularisation in this case? Because big neural networks have more capacity, so they can memorize the training data more easily (i.e. they can over-fit the training data). If the training data is not representative of the whole data distribution (that you are trying to capture with your neural network), then your neural network may fail on other data from that data distribution, i.e. it may not generalize. Regularization techniques, such as $L_1$ and $L_2$ penalties and dropout, limit the complexity of the functions that can be represented by your neural network (remember that, for a specific set of weights $\theta$, a neural network represents a specific function $f_\theta$), so they prevent your neural network from learning a complicated function that would just over-fit the training data.
Of course, you can also use regularisation when you have a very large dataset or your neural network is not that big. However, in principle, the case mentioned above is the case where you will likely need some kind of regularisation. So, as a rule of thumb, the bigger your NN is and the smaller your training dataset is, the more likely you will need some kind of regularisation.
The typical way to visualize that your neural network is over-fitting is to look at the evolution of the loss function (a type of objective function, which you want to minimize) for the training dataset and the validation dataset (a dataset that you do not use for training, i.e. updating the weights of the neural network). If the training loss is very small, while the validation loss is bigger (stays constant or even increases), as you train more the neural network, that's a good sign of over-fitting, and it suggests that you may need some kind of regularisation. However, note that, even with regularisation, it is not guaranteed that you will find a good set of weights that is able to generalize to all data (not seen during training), but it's more likely.
There are other forms of regularization, such as the KL divergence in the context of Bayesian neural networks or variational auto-encoders, but these are more advanced topics that you don't need to know now. In any case, the KL divergence has the same role as the other regularisation techniques that I mentioned above, i.e., in some way, it restricts/limits the possible functions that you can learn.


In DQN, would it be cheaper to have $N$ neural networks with a single real-valued output, one for each of the $N$ actions? In the classical examples of deep q-learning, I often see neural networks in which the input represents the state of the agent, while the output is a tuple with all the values of $Q(s, a)$ predicted for all the possible $N$ actions.
Would it be cheaper to have $N$ neural networks with a single real-valued output, one for each of the $N$ actions?
With cheaper I mean cheaper in terms of the time complexity of a single training step of the network.

Would it be cheaper to have $N$ neural networks with a single real-valued output, one for each of the $N$ actions?

I think the "No Free Lunch" theorem applies here, or something like it.
Your proposed architecture would be an unusual choice in many cases, but might be more efficient in others. For instance, it could be more efficient in the following scenario:
The long term value is highly dependent on the immediate action choice, and in a way that relies on state variables differently, depending on the specific action. That means it would be difficult for a single NN to create shared features in its layers, and you could save processing by treating each action as a different prediction problem.
This is only an educated guess.
As usual, the only way to find out for sure is to try different approaches and compare them. I don't think there is anything other than experience and a little intuition to guide you.


Is a deep technical understanding of neural networks required outside of research? To understand the inner workings of neural networks, a fair amount of mathematical concepts is required. Backpropagation alone is a challenging technique if you are not fluent in calculating local gradients. And that's just the start of the journey.

But the more I study neural networks, the more I get the impression that all those difficult mathematical concepts are only required if you are doing actual research in neural networks or want to know what's happening under the hood. If you "just" want to implement an AI utilizing a neural networks, there are several high level programming frameworks and libraries readily available including model zoos for state of the art neural networks (e.g. VGG, GoogLeNet and ResNet), that can be used.

So my question is, does a developer require a deep understanding of all the details nowadays, or have we reached a level, where frameworks take care of those details for us?

It depends on what exactly you want to be. You don't need to be mathematician if you just want to run neural networks. Most data scientists don't understand the mathematics, but they know how to run machine learning frameworks. Generally, only PhDs understand the mathematics.

In the industry, most job positions in machine learning (e.g software engineer, big data engineer etc) don't require advanced mathematics. But you should be very comfortable in programming, and understand simple things like matrix manipulation. Data engineers in the industry generally have good programming skills, but they can't read mathematical equations (or just simple equations).

However, quantitative positions in machine learning do require significant mathematics. That happens both in research and industry.

Good news --- if you don't have a PhD (e.g statistics) nobody would expect you understand the mathematics.

Summary:

If your job is more like a development position, mathematics is an advantage but not absolutely necessary
If your job is research, you do need advanced mathematics understanding (e.g. PhD)
If your job is not research but still quantitative based. Mathematics is very important.

PS: I work in research. Everybody I know who works in machine learning has a PhD.


Can neurons in MLP and filters in CNN be compared? I know they are not the same in working, but an input layer sends the input to $n$ neurons with a set of weights, based on these weights and the activation layer, it produces an output that can be fed to the next layer.
Aren't the filters the same, in the way that they convert an "image" to a new "image" based on the weights that are in that filter? And that the next layer uses this new "image"?

tl;dr The equivalent to a neuron in a Fully-Connected (FC) layer is the kernel (or filter) of a Convolution layer
Differences
The neurons of these two types of layers have two key differences. These are that the convolution layers implement:

Sparse connectivity, i.e. each neuron is connected only to an area of the input, not the whole.
Weight sharing, i.e. similar connections end up having the same weights. This is usually visualized as the same filter traversing the image.

Besides these two key differences, there are some other technical details, e.g. how the biases are implemented. Other than that they perform the same operation.
What causes some confusion is that the input of a CNN is usually 2 or 3-dimensional, while a FC is usually 1-dimensional. These aren't mandatory however. To better help visualize the differences between the two I made a couple of figures illustrating the differences between a conv-layer and a FC one, both in 1D.
Sparse connectivity
On the left are two FC neural networks. On the right, are layers with sparse connections.

Weight sharing
On the left is a sparsely connected network. The colors represent the different values of the weights. On the right is the same network with weight-sharing. Note that similar weights (i.e. arrows with the same direction in each layer) have the same value.

To answer your other questions:

Are filters not the same in the way that they convert an "image" to a new "image" based on the weights that are in that filter? And that the next layers use these new "images"?

Yes, if the input of a convolution layer is an image, so will the output. The next layer will also operate on an image.
However, I'd like to note that not all convolution layers accept images as their inputs. There are 1D and 3D convolutional layers as well.


How would 1D gradient descent look like? We have always known that gradient descent is a function of two or more variables. But how can we geometrically represent gradient descent if it is a function of only one variable?

For a function of one variable, there are only two options for directions in the domain: left or right, so it becomes almost trivial, but you can still talk about gradient descent.

You would take steps to the left if the slope/derivative is positive and make steps to the right if the slope/derivative is negative--i.e. the opposite direction of the derivative (the 1d version of the "gradient" in gradient descent), which is equivalent to the higher dimensional case.


Zero shot learning available labels in testing set As we all know, zero shot learning involves a model predicting classes that it has not seen. But we are given all the attributes each class might have.
Is it fair to assume that we are "aware" of all the class labels a dataset might have ? (Including the test set)

The formal definition of zero-shot learning is that given labeled training instances $D_{tr}$ belonging to the seen classes $S$, the aim is to learn a classifier $f^u(·):X→U$ that can classify testing instances $X_{te}$ (i.e., to predict $Y_{te}$) belonging to the unseen classes $U$.
The general idea of zero-shot learning is to transfer the knowledge contained in the training instances $D_{tr}$ to the task of testing instance classification. So it is considered a transfer learning method, and, more specifically, heterogeneous transfer learning with different label spaces.
However, since no labeled instances belonging to the unseen classes are available,
to solve the zero-shot learning problem, some auxiliary information is necessary. Such auxiliary information should contain information about all of the unseen classes. This auxiliary information usually contains some semantic information about the unseen classes, and its representation belongs to a space that is often referred as $semantic$ $space$. In the semantic space, each class has a corresponding vector representation, which is referred to as the class prototype or $prototype$ for short.
So, you should be aware of classes of your problem, in order to construct the semantic space and their class prototypes. I also suggest you read the survey paper "A survey of zero-shot learning: settings, methods and applications".


How does back-propagation through time work for optimizing the weights of a bidirectional RNN? I am aware that back-propagation through time is used for training the recurrent neural network. But I am not able to understand how this happens for the bi-directional versions of the recurrent neural networks?
So, I was hoping if anyone help me with:

Understanding with an example the training of bi-directional recurrent neural networks using back-propagation through time? (I tried following the original paper https://ieeexplore.ieee.org/document/650093, but it was kind of confusing for me when they perform the backward pass for training)

I have not implement the backprop of a bi-directional RNN from scratch so I can't be sure my answer is correct but I hope it helps.
You can see how bi-directional RNN works from this video from Andrew NG. I got the image below from that video:

For more clarity:

So if you know how to backprop through a simple RNN, you should be able to do so for bi-directional RNN.
If you need more detail, let me know.


LSTM in reinforcement learning Please tell me that is the LSTM network for the problem of reinforcement learning, as I explain to her what she will get the reward of a prediction, because the output will contain only actions?

Well, well, let's say at first I can play and upload my actions to training so that she sees which actions are right, that is, which she should strive for, but how to make her learn relatively independently?

You can use LSTM in reinforcement learning, of course. You don't give actions to the agent, it doesn't work like that.

The agent give actions to your MDP and you must return proper reward in order to teach the agent. For example if you implement trading bot, the policy(policy=the agent, which is your LSTM network) will say that at step T it is going to have action 34, which means something to your MDP and you return reward for example -0.03 or +0.05 or whatever depending what that actions is doing at the moment T.

So I get the question like you want to do a supervised learning on a reinforcement learning environment.

You can mimic supervised learning as well, but the idea of reinforcement learning is not that.

Here is how to mimic:

Scenario: you are at step T, lets say you have 3 possible actions -1,0,+1;

In a supervised learning you must give the desired action to the learning process.
In reinforcement learning you must give reward based on if you are happy or not from the agent's action.

So you must have predefined that for -1 you are not happy and you give reward 0.0, for action 0 you are not happy and you give reward 0.0 and for action +1 you are happy and you give reward +100;

I hope this makes things clear.


What is the purpose of storing the action $a$ within an experience tuple? From what I understand, experience replay works by storing tuples of $(s, a, r, s')$  to be sampled for training. I understand why we store $s$, $r$ and $s'$. However, I do not understand the need for storing the action $a$.
As I recall, the reward $r$ and the next state $s'$ are both used to calculate the target values. We can then compare these target values to the output we get when we do a forward-pass using state $s$ It seems to me that the stored action $a$ is not required for this process to work; or am I missing something? Why would we store the action $a$ if it isn't used in the training process itself?
Please, forgive me if this question has been answered before. I looked, but was unable to find anything other than generic explanations as to what experience replay is and why we do it.

We need to store the action $a$ as it tells us the action that we took in the state that we are backing up.
Suppose we are in state $s$ and we take action $a$, then we will receive a reward $r$ and next state $s'$. The goal of RL, and in particular DQN (I mention DQN as it is the first algorithm that comes to mind when I think of a replay buffer but it is of course not the only algorithm to make use of one), is that we are trying to learn optimal state-action value functions $Q(s, a)$. We thus want our value function to be able to predict $y = r + \gamma \max_{a'}Q(s', a')$, i.e. given $s$ and $a$ we want to be able to predict $y$. As you can see, we need to know which action we took in state $s$ so that we can train our value function to approximate $y$, and the value function clearly depends on $a$, hence we need to also store $a$ in our experience tuple.


Understanding the pseudocode of uniform-cost search from the book "Artificial Intelligence: A Modern Approach" On page 84 of Russell & Norvig's book "Artificial Intelligence: A Modern Approach Book" (3rd edition), the pseudocode for uniform cost search is given. I provided a screenshot of it here for your convenience.

I am having trouble understanding the highlighted line if child.STATE is not in explored **or** frontier then

Shouldn't that be
if child.STATE is not in explored **and** frontier then

The way it's written, it seems to suggest that if the child node has already been explored, but not currently in the frontier, then we add it to the frontier, but I don't think that's correct. If it's already been explored, then it means we already found the optimal path for this node previously and should not be processing it again. Is my understanding wrong?

I think this is a problem with missing brackets in pseudocode — clearly the state is only added to the frontier if it hasn't been explored already, so it would be:

if not [contains(frontier, state) OR contains(explored, state)] then

which is equivalent to your interpretation of

if not [contains(frontier, state)] AND not [contains(explored, state)] then

(according to De Morgan's Laws)

Being half-way between a programming language and natural language, this is a case where pseudocode is not quite precise enough.


Given an image and two points $A$ and $B$ on that image, how could we find a path from $A$ to $B$? If we have a search or path-finding problem, A* and Dijkstra's algorithm require that we formulate it as a search in a graph with nodes and connections between these nodes. If there are obstacles, we also need to encode this information in the graph, so that they are not traversed. Additionally, there may be costs/weights on the connections between points. If such weights/costs are high, the algorithms won't take that path.
I've been using A* and Dijkstra's algorithm this way so far. However, it's a bit cumbersome to always have to define the nodes/points and the relationships (or connections) between them. There's no learning here. I just define a graph and the algorithms search on this graph.
Let's say I have a white image, a green blob in the middle, and points $A$ and $B$ at either side of the blob, I need to get from $A$ to $B$. I don't have a search space represented as a graph here. I just have this image.
Could I use machine learning to solve this problem (and would it generalize to more complex maps)? If so, are there any research works on this topic? Or is that the wrong route to take (pardon the pun)?

Let's say I have a white image, a green blob in the middle, and points $A$ and $B$ at either side of the blob, I need to get from $A$ to $B$. I don't have points to traverse. I just have this image.

If your image is as "simple" as you describe here, with very easily distinguishable colours, the easiest solution would likely be to construct a graph as expected by algorithms such as A* as follows:

Every pixel becomes a node in the graph, with connections to all adjacent pixels
Green pixels (the green blob in the middle) are marked as non-traversable (or simply don't have connections to adjacent pixels) (I assume this thing is your obstacle).
Pixels belonging to the points on either side of the obstacle are marked as starting and goal vertices.

Then, you have the graph that a traditional pathfinding algorithm like A* would expect, and can simply run the algorithm there.

If you get less "clear" images (i.e. not brightly coloured, easily distinguishable things, but more like real-life top-down images of an area), then the above won't really work anymore. In such cases, you'd likely want to look into Reinforcement Learning approaches, where you give an agent a reward when it successfully manages to find a connection between start and goal. In particular, you'd need Deep Reinforcement Learning approaches, because otherwise you're not going to be able to handle complex images.
Note that these approaches are not going to be simple to implement or immediately understand for 100% if you're a beginner. There won't be any "beginner-friendly" approaches for the problem you describe, it simply is quite a complex problem (requiring a mix of "decision-making" and "understanding" images).


How do I predict the occurrence of rare events? I am trying to predict crime. I have data with factors: location, keyword description of the crime, time crime occurred and so on. This is for crimes that occurred in the past.

I would like to treat the prediction of crimes as a binary classification problem. In this model, the data I have collected would form the "positive" examples: they are all examples of a crime happening. However, I am unsure what to use for the negative examples.

Obviously, most of the time there is no crime at the location, but can I use this as negative data? For example, if I know there was a crime at 7pm at location X, and no other crimes there, should I generate new negative data points for every hour except 7pm?

Ideally, I want to create probabilities of crime based on a set of factors.

It might be more informative to:

Label each combination of location, type, and time of crime with a crime rate. For example, theft, in Crystal City, at 11pm at night, occurs 20 times per year, or 0.4 times per resident per year.
Predict the crime rate, rather than individual events.

This avoids the need to have explicit examples of "non-crime", and lets you instead directly learn something related to the probabilities of crimes being committed (the rate).


Should I use neural networks or genetic algorithms to solve Gomoku? Currently, I'm doing a project that's about creating an AI to play the game Gomoku (it's like tic tac toe, but played on a 1515 board and requires 5 in a row to win). I have already successfully implemented a perfect tic tac toe AI using Q-learning and having game states/actions stored in a table, but, for a 1515 board, the possible game states become too large to implement this project.
My question is, should I use neural networks or genetic algorithms for this problem? And more specifically, how should I implement this?

For Gomoku, it seems a bit of an overkill to use neural networks or the genetic algorithm as both take a while, and more often than not, don't go how you want it to. The Gomoku game tree is rather large, but you can get a decent AI from minimax, game tree pruning, and a good heuristic function (that includes counting half and full 2s,3s,4s,...etc.) as opposed to mapping out the full space.
If you are unfamiliar with alpha-beta pruning and minimax see this article.
If you really want to use neural networks or genetic algorithms you can for the learning experience. Regarding neural networks, one way to do it is the following:

Define a heuristic function that receives a board state input (sequence of 0,1,2 for empty, black, white), and outputs a 'goodness' value of the board state. The neural network is our heuristic function.
Assuming that the moves in these game are optimal, train off of the difference between the current best move (by your current parameters) and what move your data says is the best. This is how we define our error function! Thus you are minimizing this difference so that what move your neural network says is the strongest is ideally what your game data says is the strongest (Optimization of this error function can be done via backpropagation or genetic algorithm).
Ideally, by this point, you can now use your ('strong') neural network-based evaluation function for your game tree move evaluations instead of hardcoded heuristics.

Of course, this is just one way, and you would need to find game data first.
As a side note, applying a genetic algorithm can occur in a few ways, such as parameter optimization in a neural network as mentioned above or game tree searching, so make sure you are clear how you define the problem setting with it! The same goes for alternative ways to apply a neural network.
Finally, it's helpful to know Gomoku is solved. See this post for others' thoughts and ideas.


Has there ever been a model (neuro-physical) proposed for the human unconscious brain? Has there ever been a model (neuro-physical) proposed for the human unconscious brain?

I tried reading on unconscious mind but couldn't find anything which answers the question.

Is there any literature related to it?

The Unconscious in psychology is related to many aspects of behavior, but many levels of this exist, including low levels (simple tasks) like walking, eating, breathing etc.  This has been investigated in animal world, for example muscular memory, the insect brain model, and Nonlinear System Control Using Neural Networks . Many of these are related to body control.

See also: nested hierarchies of circular emotional control and cognitive regulation for “bottom-up” influences and “top-down” regulations


Using neural network to recognise patterns in matrices I am trying to develop a neural network which can identify design features in CAD models (i.e. slots, bosses, holes, pockets, steps).

The input data I intend to use for the network is a n x n matrix (where n is the number of faces in the CAD model). A '1' in the top right triangle in the matrix represents a convex relationship between two faces and a '1' in the bottom left triangle represents a concave relationship. A zero in both positions means the faces are not adjacent. The image below gives an example of such a matrix.

Lets say I set the maximum model size to 20 faces and apply padding for anything smaller than that in order to make the inputs to the network a constant size.

I want to be able to recognise 5 different design features and would therefore have 5 output neurons - [slot, pocket, hole, boss, step]

Would I be right in saying that this becomes a sort of 'pattern recognition' problem? For example, if I supply the network with a number of training models - along with labels which describe the design feature which exists in the model, would the network learn to recognise specific adjacency patterns represented in the matrix which relate to certain design features?

I am a complete beginner in machine learning and I am trying to get a handle on whether this approach will work or not - if any more info is needed to understand the problem leave a comment. Any input or help would be appreciated, thanks.

Would I be right in saying that this becomes a sort of 'pattern recognition' problem?

Technically, yes. In practice: no.

I think you might be interpreting the term "pattern recognition" a bit too literal. Even though wikipedia defines Pattern recognition as "a branch of machine learning that focuses on the recognition of patterns and regularities in data", it's not about solving problems that can "easily" be deduced by logical reasoning.

E.g. you say that

  A '1' in the top right triangle in the matrix represents a convex relationship between two faces and a '1' in the bottom left triangle represents a concave relationship

This is true always. In a typical machine learning situation, you wouldn't (usually) have this prior knowledge. At least not to the extent that it would b be tractable to “solve by hand”.

Pattern recognition is conventionally a statistical approach to solving problems when they get too complex to analyze with conventional logical reasoning and simpler regression models. Wikipedia also states (with a source) that pattern recognition "in some cases considered to be nearly synonymous with machine learning".

That being said: you could use pattern recognition on this problem. However, it seems like overkill in this case. Your problem, as far as I can understand, has an actual "analytical" solution. That is: you can, by logic, get a 100% correct result all the time. Machine learning algorithms could, in theory, also do this, and in that case, and this branch of ML is referred to as Meta Modelling[1].

  For example, if I supply the network with a number of training models - along with labels which describe the design feature which exists in the model, would the network learn to recognise specific adjacency patterns represented in the matrix which relate to certain design features?

In a word: Probably. Best way to go? Probably not. Why not, you ask?

There is always the possibility that your model doesn't learn exactly what you want. In addition you have many challenges like overfitting that you'd need to concern yourself about. It's a statistical approach, as I said. Even if it classifies all your test data as 100% correct, there is no way (unless you check the insanely intractable maths) to be 100% sure that it will always classify correctly. I further suspect that you're also likely to end up spending more time working on your model then the time it would take to just deduce the logic.

I also disagree with @Bitzel: I would not do a CNN (convolutional neural network) on this. CNNs are used when you want to look at specific parts of the matrix, and the relation and connectedness between the pixels are important — for example on images. Since you only have 1s and 0s, I strongly suspect that a CNN would be vastly overkill. And with all the sparsity (many zeros) you’d end up with a lot of zeros in the convolutions.

I'd actually suggest a plain vanilla (feed forward) neural network, which, despite the sparsity, I think will be able to do this classification pretty easily.