Machine learning to detect wrong address data I want to use a machine learning algorithm to detect false address data. I learned about neural networks and machine learning at university, but I don't have much experience in this field.

Do you think it is feasible to use a high level algorithm for this or should I use simple queries and filters to catch out wrong data?

The comments are off base.  Having worked in validation of data as a consultant for Nasdac, Amex, and Lexis Nexis I can tell you that using the UNIX sed -r or pcrelib is insufficient to do a stellar cleansing of address data.

Although none of those companies did this at the time I was consulting, the application of current machine learning is easy to infer from the basic characteristics of data.  What is needed is a good record of incoming data, cleaned data, and rejected records to constantly use as a reference.

The manual process would be a nightmare.  For instance, misspellings are more indicative of true data than properly spelled ones in some but not all cases (unless the falsified data generator is programmed to synthesize the existing distribution of misspellings in authentic data sources).

Try to profile that in static code such that the data validation will adapt to data trends without requiring a maintenance workforce.  Those kinds of things never get addressed in a typical IT environment.  Non-adaptive validation would be a gross mistake, especially if the volume is high, like for M16 or Liberty Cross or the NSA or Interpol.

What you want is an extremely fast index of well authenticated good addresses (perhaps corroborated by public records with local authentication policies) and a feature extraction from it using auto encoding or some similar and perhaps better methodology and its associated algorithms.

Then you can train the classification of fake and authentic addresses based on extracted feature profiles.

You will also want reinforcement, because the attackers (those trying to create false identities using plausible addresses) will adapt to the training of the current system.  The authentication system must stay steps ahead of those trying to defeat it, which they will likely try to do once the existence of authentication automation is detected.

One can fend of attackers by placing misinformation strategically and then tracing the input sources back to the attacker based on the misinformation seeds.  That only works if you have law enforcement in your camp.


What is the proof behind the gradient of a curve being proportional to the distance between the two co-ordinates in the x-axis? In the [delta rule][1] the equation to adjust the weight with respect to error is
$$w_{(n+1)}=w_{(n)}-\alpha \times \frac{\partial E}{\partial w}$$
*where $\alpha$ is the learning rate and $E$ is the error.
The graph for $E$ vs $w$ would look like the one below with $E$ in the $y$ axis and $W$ in the $x$-axis

In other words, we can write
$$\alpha \times \frac{\partial E}{\partial w}=w_{(n)}-w_{(n+1)}$$
I want to know, what is the proof behind the gradient of a curve being equal/proportional to the distance between the two coordinates in the x-axis.
$\frac{\partial E}{\partial w}$ times step is a small shift on $f(w)$ not $w$. So, why does the difference between $W(n+1)$ and $W(n)$ be equal to $f(W)$?
I found a similar question, but the accepted answer doesn't have a proof.

Don’t think about it as the $w_{(n)}-w_{(n+1)}$ being proportional to something. Think about it this way:
I'm now at $w_{(n)}$. Where do I want to be at timestep, so that the error decreases? For that, I need to know how the error changes when I make small steps to the left or right of $w_{(n)}$.
If $E$ increases as I increase $w$ (that is, if $\frac{\partial E}{\partial w}>0$, then obviously, I would want to move a little bit to the left. In other words, $w_{n+1}<w_{n}$ or $w_{n+1}-w_{n}<0$.
On the other hand, if the derivative were negative, you know that you should move right to reduce the error a little bit, $w_{n+1}-w_{n}>0$.  So, basically, your step should have the opposite sign of the derivative.
$$w_{n+1}-w_{n} \propto-\frac{\partial E}{\partial w}$$
$\alpha$, the learning rate, is just the constant of proportionality.
Caution: think about small values for this rate, not big numbers.  Taking a huge step can cause you to overshoot the minimum point.


How can I determine the mathematical relation between the input and output variables? I would like to take in some input values for $n$ variables, say $R$, $B$, and $G$. Let $Y$ denote the response variable of these $n$ inputs (in this example, we have $3$ inputs). Other than these, I would like to use a reference/target value to compare the results.

Now, suppose the relation between the inputs ($R$, $B$ and $G$) with the output $Y$ is (let's say):

$$Y = R + B + G$$

But the system/machine has no knowledge of this relation. It can only read its inputs, $R$, $B$ and $G$, and the output, $Y$. Also, the system is provided with the reference value, say, $\text{REF} = 30$ (suppose).

The aim of the machine is to find this relation between its inputs and output(s). For this, I have come across some quite useful material online like this forum query and Approximation by Superpositions of a Sigmoidal Function by G. Cybenko and felt that it were possible. Also, I doubt that Polynomial Regression may be helpful as suggested Here.

One vague approach that comes to my mind is to use a truth table like approach to somehow deduce the effect of the inputs on the output and hence, get a function for it. But neither am I sure how to proceed with it, nor do I trust its credibility.

Is there any alternative/already existing method to accomplish this?

There are always a large number of possible functions that can produce a given set of input-output values. The challenge is to find a simplest function (according to whatever criteria you choose) to produce those values.

One approach is to write a general function of the input variables, comprising terms of all order in R, G, and B, with a coefficient for each term, then search for values of the coefficients that A) reproduce the known input-output values accurately and B) leave the largest number of coefficients equal to zero.

Several different algorithms can be used to do the search efficiently.  My choice would be a genetic algorithm to seek the minimum of the RMS difference between the produced and known I-O values, summed with the product of a gradually increasing parameter and the number of nonzero coefficients.


Are more than 8 high performance Nvidia GPUs practical for deep learning applications? I was prompted towards this question while trying to find server racks and motherboards which are specialized towards artificial intelligence. Naturally I went to the SuperMicro website. There the chassis+motherboard which supported the maximum GPUs in the "artificial intelligence" category could support upto 8 of them. Additionally, the Nvidia DGX-1 also has only 8 Tesla P100 GPUs. And just to rub it in, Matlab does not support more than 8 GPUs last I checked.

So, are more than 8 GPUs practical for DL? I would take Caffe, CNTK, Tensorflow and Torch7 as reference.

I did some recent research on this topic. It all comes down to parallelization.
Basically there are 2 ways to do it: model parallelization or batch parallelization.

Model parallelization is when you split the model by layers among multiple GPUs. As per my best knowledge you can't split a layer between GPUs, so 8 GPUs would serve 8 layers that is very extensive. Tensorflow supports this method. In my opinion more than 6 doesn't make sense this way.

Batch parallelization is when you run the entire model on each GPU parallel and you split your batch and process it in parallel. This is done via a trick to define a larger batch that will be split and become the desired batch size after split. In this case batch splitting and updating the weights is done on the CPU (in case of Tensorflow) and after 3 GPU any additional GPU has only a marginal improvement on training speeds (as per reports). So here even 4 doesn't make sense and 8 is just crazy.  Here is an example of batch parallelization.

Alternately if you are good in coding you may want to have a look at this paper, section 3.5 where it is explained how 8 GPUs were utilized to serve a 4 layer LSTM network. Probably you can do things like this to utilize DGX-1 but as far as I know Tensorflow doesn't support splitting a layer to multiple GPUs. My conclusion is that it's already very hard to utilize 8 GPUs and above that bus speed becomes the bottleneck.

Extension:
I double checked on the bus speed and I was wrong, it should not be a problem. Most of the time consumed in training is computational effort of backpropagation.

Actually PCIe speed is limited by the CPU and the mobo chipset. The CPU has PCIe lanes to be allocated by the mobo. The strongest single CPU at the moment is Broadwell-E with 40 lanes (Skylake rumored to have 44). Mobo allocates that bandwidth either x16 or x8 to PCIe peripherals. So with a 40-lane CPU you can run 2 cards at x16 (2 * 16 = 32 < 40) or 5 cards at x8 (5 * 8 = 40). Here it needs to be mentioned that M.2 also uses PCIe lanes so for the latter option forgot and M.2 drive. A single CPU system will not take 8 GPUs, that's why they need dual CPU in DGX-1. The next limitation is the mobo chipset. The most powerful at the moment is the X99 and the C6 series. X299 will be announced next week if memory serves and probably a C6 replacement follows soon.

Since PCIe speed is not the bottleneck in machine learning the updated answer for your question would be the more GPUs the better. It seems the limit is 5 at x8 using PCIe 3.0, but more than that is rarely needed since layers can't be split between GPUs. (And the deepest NN that still makes sense may be considered as 5-6 layers.)


What is the scope of real-world deep learning applications in 2020? 2015 was a milestone year for AI--"deep learning" was validated in a very public way with AlphaGo.  However, at the time, the question was raised: "What else is deep learning good for?"
5 years later, I want to gauge:

How is deep learning applied to real world problems in 2020?  What real world applications is it currently used for?

Deep learning is used to perform language translation in Google Translate [1]. Specifically, Google Translate now uses transformers and RNNs rather than the original GNMT system (proposed in 2016), which was also based on neural networks. Deep learning is also used in DeepL (though I cannot find a good resource to cite apart from Wikipedia [2] given that the system is closed-source), a strong alternative to Google Translate. However, note that, in general, machine translation is still far from perfect and it is probably not adopted to perform serious translations.
Tesla's autopilot also uses neural networks [3].

Built on a deep neural network, Tesla Vision deconstructs the car's environment at greater levels of reliability than those achievable with classical vision processing techniques.

DeepFakes are also developed using deep learning techniques [4].


What is the relationship between gradient accumulation and batch size? I am currently training some models using gradient accumulation since the model batches do not fit in GPU memory. Since I am using gradient accumulation, I had to tweak the training configuration a bit. There are two parameters that I tweaked: the batch size and the gradient accumulation steps. However I am not sure about the effects of this modification so I would like to fully understand what is the relationship between the gradient accumulation steps parameter and the batch size.
I know that when you accumulate the gradient you are just adding the gradient contributions for some steps before updating the weights. Normally, you would update the weights every time you compute the gradients (traditional approach):
$w_{t+1} = w_t - \alpha \cdot \nabla_{w_t}loss$
But when accumulating gradients you compute the gradients several times before updating the weights (being $N$ the number of gradient accumulation steps):
$w_{t+1} = w_t - \alpha \cdot \sum_{0}^{N-1} \nabla_{w_t}loss$
My question is: What is the relationship between the batch size $B$ and the gradient accumulation steps $N$?
By example: are the following configurations equivalent?

$B=8, N=1$: No gradient accumluation (accumulating every step), batch size of 8 since it fits in memory.
$B=2, N=4$: Gradient accumuluation (accumulating every 4 steps), reduced batch size to 2 so it fits in memory.

My intuition is that they are but I am not sure. I am not sure either if I would have to modify the learning rate $\alpha$.

There isn't any explicit relation between the batch size and the gradient accumulation steps, except for the fact that gradient accumulation helps one to fit models with relatively larger batch sizes (typically in single-GPU setups) by cleverly avoiding memory issues. The core idea of gradient accumulation is to perform multiple backward passes using the same model parameters before updating them all at once for multiple batches. This is unlike the conventional manner, where the model parameters are updated once every batch-size number of samples. Therefore, finding the correct batch-size and accumulation steps is a design trade-off that has to be made based on two things: (i) how much increase in the batch-size can the GPU handle, and (ii) whether the gradient accumulation steps result in at least as much better performance than without accumulation.
As for your example configurations, there are the same in theory. But, there are a few important caveats that need to be addressed before proceeding with this intuition.

Using Batch Normalization with gradient accumulation generally does not work well, simply because BatchNorm statistics cannot be accumulated. A better solution would be to use Group Normalization instead of BatchNorm.
When performing a combined update in gradient accumulation, it must be ensured that the optimizer is not initialized to zero (i.e. optimizer.zero_grad()) for every backward update (i.e. loss.backward()). It is easy to include both statements in the same for loop while training which defeats the purpose of gradient accumulation.

Here are some interesting resources to find out more about it detail.

Thomas Wolf's article on different ways of combating memory issues.
Kaggle discussions on the effect on learning rate: here and here.
A comprehensive answer on gradient accumulation in PyTorch.


What are evolutionary algorithms for topology and weights evolving of ANN (TWEANN) other than NEAT? I wonder, if there are other than NEAT approaches to evolving architectures and weights of artificial neural networks?

To be more specific: I am looking for projects/frameworks/libraries that use evolutionary/genetic algorithms to simultanousely evolve both topology and train weights of ANNs other than NEAT approach. By 'other' I mean similar to NEAT but not based entirely on NEAT. I hope to find different approaches to the same problem.

The Wikipedia article on neuroevolution contains a list of neuroevolution techniques (e.g. NEAT). I will list below the examples that evolve both the parameters and the topology of the neural network.

Cellular encoding (1994)
GNARL (1994)
EPNet (1997)
Evolutionary acquisition of neural topologies (2007)
Interactively constrained neuro-evolution (2012)
Deus ex neural network (2012)
Spectrum-diverse unified neuroevolution architecture (2017)
Covariance matrix adaptation with hypervolume sorted adaptive grid algorithm (2017)


How can I translate a natural language question to an MDX query? I am researching Natural Language Processing (NLP) to develop an NL Question Answering system. The answering part is already done. So processing the question remains, along with the questions regarding the algorithms.
The final product should allow the user to ask a question in NL. The question then gets translated to an Multidimensional Expressions (MDX) query, which generates a script regarding the dimensions of the cube.
How can I translate a natural language question to an MDX query?
The outcome of the question is in the form of a calculation. For example

How many declarations were done by employee 1?

or

Give me the quantities for Sales.

You can use a component library which can help you to implement Natural language query builder in your application( the question part ) called Open Natural Language Processing Package
 ,  so you can definitely develop a module, by using existing modules of OpenNLP such as entity extraction, chunking and parsing.

According to wikipedia source   ; it points out that as of 2001, Q&A applications typically includes "a question classifier module which determines the type of question and the type of answer," so "a multiagent question and answering architecture has been proposed, where each domain [or variable] is represented by an agent which tries to answer questions, taking into account its specific knowledge."

But it still need some effort to build a NLgenerators inline with databases , for the answer query and also is the link to help you on how you can work with Compositional Semantic Parsing on Semi-Structured Tables

Hope this can give you some insight.


Seeking advice: Building a NLP in C++ Context:
I'm taking an advanced C++ course and there is a large 2 month project of my choosing that I'll be starting soon.
I want to build a NLP that can do text completion, but I'm new to this area and would love some direction from experienced humans. I'm thinking something along these lines:
user input: "Looking through her telescope ..."
program output: "...she saw Jupiter and the Galilean moons."

I'm a reasonably competent programmer with intermediate c++ experience, but I've only built a couple very basic neural networks before (both in python).
Questions:

What do you think is achievable in 2 months? Is full sentence completion reasonable (like in my above example), or should I drop down to only one word completion (eg. input: "I took off my shoe and removed the...", output: "pebble.")? Other related ideas welcomed!
Ideally I wouldn't be specifying any grammar rules, just training the network on a large text dataset. What sort of ML structure would be best for this?
What libraries or datasets would you recommend for a project like this?
Any papers or reading resources you'd recommend for me to better understand this territory?

I think this is a risky project. There is a lot of specialist knowledge in neural networks and NLP. Trying to learn it "on the side" in order to complete your project may be biting off more than you can chew.
If you intend to write the neural network and train it yourself, you should be prepared for long sessions of training and testing variations of the neural network entirely separate from practicing any of the advanced C++ code that you are studying.
My recommendation instead of writing your own NN components, would be to learn the C++ API for Tensorflow and reference examples (like this one in Python/Keras) of using neural networks built in Tensorflow specifically for predicting sequence continuation.
I would also suggest you spend a little time trying out the Python and Keras-based networks to get a feel for what NN architectures work for your problem. If you use Python's tensorflow bindings, you may even be able to train a neural network separately using someone else's script. After training, in your C++ project you could then load the NN definition and use it for the NLP part of the project. You can search for examples of using C++ Tensorflow to load saved networks - this is a relatively common use case when someone wants to re-use a trained network for making new predictions on some edge device where the speed and efficiency of C++ make a large difference.


Is there an AI model with "certainty" built in? If I see a hundred elephants and fifty of them are grey I'd say the probability of an elephant being grey is 50%. And my certainty of that probability is high.

However, if I see two elephants and one of them is grey. Still the probability is 50%. But my certaintity of this is low.

Are there any AI models where not only the probability is given by the AI but it's certainty is also?

"Certainty" might be thought of as the probability that the probability is correct.

This could go up more levels.

Is there any advantage in doing this?

One way I can envisage this working is instead of a weight, the NN stores two integers $(P,N)$ which represent positive and negative evidence and the weight is given by $P/(P+N)$. And each iteration $P$ or $N$ can only be incremented by 1.

You make a valid point, vanilla neural networks cannot give you more than a point estimate of class confidence. If one wanted to actually gain an idea of variance, you need a framework that allows such a mechanism.

A popular methodology to this is Bayesian modeling. In other words given some data, $\Omega$, you want to create some form of descriminative model $p(y|x;\theta)$ where $\theta$ denotes the parameters that are random variables. This is unline NN's where they also learn some descriminative model $p(y|x;\theta)$ where $\theta$ are fixed. This difference is key to your goal, because if $\theta$ is fixed, $Var(Y|X) = 0$ since any time you put in the same $X$ into the network, itll always come out the same. On the other hand a bayesian model will have a variance, $Var(Y|X) = E[(Y - E[Y|X])^2 | X]$ and $(Y - E[Y|X])^2 | X$  is no longer gauranteed to be 0 and can be empirically measured through some MC method (or analytically based on your model).

Note that your idea is on the right track, that the more data points you have, the more confident your estimates will be, but the only issue is that just considering $\frac{P}{(P+N)}$ in a neural network will be just a heuristic and is difficult to quantify its association to the variance.

I think this may fit your fancy: high level overview of bayesian network blog post. This is a high level overview of using a neural netowrk paradigm but including uncertainty in the weights. Theres tons of tricks out there to make it so you can still even train these with gradient descent and such.


CPU preferences and specifications for a multi GPU deep-learning setup For a multi (4xTitan Xp) GPU deep learning setup what kind of CPU is preferable?

Specifically I am comparing:

Intel Xeon E5-2620 with 8x2.1GHz 20MB L3 Cache
Intel Xeon E5-1620K with 4x3.5Ghz 10MB L3 Cache
Intel Xeon E5-1650K with 6x3.6GHz 15MB L3 Cache
Intel i7-6850K with 6x3.6GHz 15MB L3 Cache

I wonder if the higher clock rates are important or is it better to have more number of cores in this use case.

Lets answer the question assuming ideal case.

Say you are training a deep learning algorithm in which p proportion of the algorithm is parllelizable and (1-p) proportion as sequential part . Lets assume you can perfectly divide the program into their parallelizable and sequential parts. I don't exactly know the specs of aforementioned CPU's, but lets assume for now they are symmetrical processors (which is the most common) i.e. all the cores or CPU's perform the task equally and there is  no master CPU coordinating the functioning between the different cores.

Now as per a law called Amdahl's Law which states:

Where p holds the same meaning, while N can be approximated as the number of cores sharing the task and S is the reciprocal of the rate of speedup you will get (that is time required will be T * S(N). So, for each particular GPU's with N number of CPU's you can calculate the times of speedup you will get.

Specifically, calculate the number of calculations your NN will make while training. Here is a quite good calculation of the same. Generally a CPU will take 6-7 clock cycles for multiplication and 60-70 for division. Calculate the total number of clock cycles taken = a by the learning algorithm. Then calculate the time taken to complete the calculation by a single CPU by the formula a/clock_speed. Then using Amdahl's law calculate the speedup where N will be the number of CPU's in the GPU.

So you see, it clearly depends on the type of program you are running. Cache memory also increases the speed of computation. How? Cache memory is basically a temporary storage memory. When the program accesses your data/observations, it first checks if the data is available in cache memory since it can access data faster from cache memory, if not available then it checks the RAM then your hard-disk and loads it first unto RAM then to cache. This is  an oversimplified version of cache working.

Also check the number of actual physical CPU's, virtual CPU's are useless in parallelization. For example i7 generally consists of 4 physical CPU's.

The part I skipped, if there exists a master CPU just replace N with N-1 since it only performs the task of coordination.

So now you just have to find the general class of programs you will  be running and check its sequential and parallelizable parts.

Hope this helps!

EDIT : Recently I learnt that not all GPU's support the most popular deep learning framework tensorflow-gpu (i.e. the package to utilise multiple GPU's). It is specifically supported by NVIDIA GPU's as the CUDA framework required for tensorflow-gpu is specifically made for NVIDIA. Also some newer other brand GPU's are supporting tensorflow-gpu. Although tensorflow CPU only version can be run on any system. (Anyone can clarify this matter if they have more knowledge about this).


What is the actual quality of machine translations? As an AI layman, till today I am confused by the promised and achieved improvements of automated translation.
My impression is: there is still a very, very far way to go. Or are there other explanations why the automated translations (offered and provided e.g. by Google) of quite simple Wikipedia articles still read and sound mainly silly, are hardly readable, and only very partially helpful and useful?
It may depend on personal preferences (concerning readability, helpfulness, and usefulness), but my personal expectations are disappointed sorely.
The other way around: Are Google's translations nevertheless readable, helpful, and useful for a majority of users?
Or does Google have reasons to retain its achievements (and not to show to the users the best they can show)?

Who claimed that machine translation is as good as a human translator? For me, as a professional translator who makes his living on translation for 35 years now, MT means that my daily production of human quality translation has grown by factor 3 to 5, depending on complexity of the source text.

I cannot agree that the quality of MT goes down with the length of the foreign language input. That used to be true for the old systems with semantic and grammatical analyses. I don't think that I know all of the old systems (I know Systran, a trashy tool from Siemens that was sold from one company to the next like a Danaer's gift, XL8, Personal Translator and Translate), but even a professional system in which I invested 28.000 DM (!!!!) failed miserably.

For example, the sentence:

  On this hot summer day I had to work and it was a pain in the ass.

can be translated using several MT tools to German.

Personal Translator 20:

  Auf diesem heißen Sommertag musste ich arbeiten, und es war ein Schmerz im Esel.

Prompt:

  An diesem heißen Sommertag musste ich arbeiten, und es war ein Schmerz im Esel.

DeepL:

  An diesem heißen Sommertag musste ich arbeiten und es war eine Qual.

Google:

  An diesem heißen Sommertag musste ich arbeiten und es war ein Schmerz im Arsch.

Today, Google usually presents me with readable, nearly correct translations and DeepL is even better. Just this morning I translated 3500 words in 3 hours and the result is flawless although the source text was full of mistakes (written by Chinese).


How could self-driving cars make ethical decisions about who to kill? Obviously, self-driving cars aren't perfect, so imagine that the Google car (as an example) got into a difficult situation.
Here are a few examples of unfortunate situations caused by a set of events:

The car is heading toward a crowd of 10 people crossing the road, so it cannot stop in time, but it can avoid killing 10 people by hitting the wall (killing the passengers),
Avoiding killing the rider of the motorcycle considering that the probability of survival is greater for the passenger of the car,
Killing an animal on the street in favour of a human being,
Purposely changing lanes to crash into another car to avoid killing a dog,

And here are a few dilemmas:

Does the algorithm recognize the difference between a human being and an animal?
Does the size of the human being or animal matter?
Does it count how many passengers it has vs. people in the front?
Does it "know" when babies/children are on board?
Does it take into the account the age (e.g. killing the older first)?

How would an algorithm decide what it should do from the technical perspective? Is it being aware of above (counting the probability of kills), or not (killing people just to avoid its own destruction)?
Related articles:

Why Self-Driving Cars Must Be Programmed to Kill
How to Help Self-Driving Cars Make Ethical Decisions

How could self-driving cars make ethical decisions about who to kill?

It shouldn't. Self-driving cars are not moral agents. Cars fail in predictable ways. Horses fail in predictable ways.

  the car is heading toward a crowd of 10 people crossing the road, so
  it cannot stop in time, but it can avoid killing 10 people by hitting
  the wall (killing the passengers),

In this case, the car should slam on the brakes. If the 10 people die, that's just unfortunate. We simply cannot trust all of our beliefs about what is taking place outside the car. What if those 10 people are really robots made to look like people? What if they're trying to kill you?

  avoiding killing the rider of the motorcycle considering that the
  probability of survival is greater for the passenger of the car,

Again, hard-coding these kinds of sentiments into a vehicle opens the rider of the vehicle up to all kinds of attacks, including "fake" motorcyclists. Humans are barely equipped to make these decisions on their own, if at all. When it doubt, just slam on the brakes.

  killing animal on the street in favour of human being,

Again, just hit the brakes. What if it was a baby? What if it was a bomb?

  changing lanes to crash into another car to avoid killing a dog,

Nope. The dog was in the wrong place at the wrong time. The other car wasn't. Just slam on the brakes, as safely as possible.

  Does the algorithm recognize the difference between a human being and an animal?

Does a human? Not always. What if the human has a gun? What if the animal has large teeth? Is there no context?

  Does the size of the human being or animal matter?
  Does it count how many passengers it has vs. people in the front?
  Does it "know" when babies/children are on board?
  Does it take into the account the age (e.g. killing the older first)?

Humans can't agree on these things. If you ask a cop what to do in any of these situations, the answer won't be, "You should have swerved left, weighed all the relevant parties in your head, assessed the relevant ages between all parties, then veered slightly right, and you would have saved 8% more lives." No, the cop will just say, "You should have brought the vehicle to a stop, as quickly and safely as possible." Why? Because cops know people normally aren't equipped to deal with high-speed crash scenarios.

Our target for "self-driving car" should not be 'a moral agent on par with a human.' It should be an agent with the reactive complexity of cockroach, which fails predictably.


How does MuZero learn to play well for both sides of a two-player game? I'm coding my own version of MuZero. However, I don't understand how it supposed to learn to play well for both players in a two-player game.
Take Go for example. If I use a single MCTS to generate an entire game (to be used in the training stage), couldn't MuZero learn to play badly for black in order to become good at predicting a win for white? What is forcing it to play well at every turn?

Both players are represented by the exact same network with the exact same weights(similar to AplhaGO, AlphaGoZero and AlphaZero). So, they will both behave identical. Because you only have a single network, MuZero can not learn two different policies, but only one.
You can also think of this in the following way: MuZero actually learn to play only with white(or black, but just one of them) without knowing to play with the other color (at least multiple implementations of the previous algorithms like AlphaGo Zero and AlphaZero that are similar to MuZero are doing exactly this). So, in order to trick it to also be able to play with the other color, when the network need to play with black, you just flip the colors on the table so that black becomes white(and white becomes black) and the network knows what to do. After choosing the move, you flip the whole thing back and that is usually how it is done. So, from the perspective of your network, it will always play white, but because you do the flipping of the colors you can actually put them to play against each other without them knowing that.
Even without using this trick of flipping the color of the table, by doing the MCTS simulations, you will have for each state the statistics for the actions, and usually as you do more simulations, this statistics show you which actions are the best in each state. And when you train, you try to imitate this. So, your network will learn in each state which actions are the best, and this is the reason why it learns to take the best possible action in each state.


Convergence in multi-agent environment I have a multi-agent environment where agents are trying to optimise the overall energy consumption of their group. Agents can exchange energy between themselves (actions for exchange of energy include - request, deny request, grant), which they have produced from renewable sources and is stored in their individual batteries. The overall goal is to reduce the energy used from non-renewable sources.

All agents have been built using DQN. All (S,A) pairs are stored in a replay memory which are extracted when updating the weights.

The reward function is modelled as such — if at the end of the episode the aggregate consumption of the agent group from non-renewable sources is lesser than the previous episode, all agents are rewarded with +1. If not, then -1. An episode (iteration) consists of 100 timesteps after which the reward is calculated. I update the weights after each episode.

The reward obtained at the end of the episode is used to calculate the error for ALL (S,A) pairs in the episode i.e. I am rewarding all (S,A) in that episode with the same reward.

My problem is that agents are unable to learn the optimal behavior to reduce the overall energy consumption from non-renewable sources. The overall consumption of the group is oscillating i.e. sometimes increasing and sometimes decreasing. Does it have to do with the reward function? Or Q learning as the environment is dynamic?

Does it have to do with the reward function?

This seems likely to me. You have chosen a reward that is unusual in that it cross-links episodes. It is not really a reinforcement problem to optimise behaviour with respect to results of previous episode behaviour in this way. This might be an option for an evolutionary fitness context, if you have competing teams against the same environment in a tournament style selection.

Reinforcement learning should really take as direct measure of your goal as you can construct. In this case you want to minimise some scalar quantity, and that is an obvious candidate as a negative reward. So the reward should be the negative of the total non-renewable energy consumption. The maximum possible value in theory for any single episode would be zero.

You may still have problems with a multi-agent setup oscillating using Q-learning, it will depend on how much each agent views the full, relevant state, and whether you are training several distinct agents at the same time (more likely to oscillate), or a single type of agent with multiple instances in each environment (less likely to oscillate, but still can if it has too blinkered a view of the environment as experienced by the other agents). But having a single shared goal with shared reward like you do here should in theory help with stability.


Why is it useful to define the return as the sum of the rewards from time $t$ onward rather than up to $t$? Why is it useful to define the return as the sum of the rewards from time $t$ onward rather than up to $t$?
The return for an MDP is usually defined as
$$G_t=R_{t+1}+R_{t+2}+ \dots +R_T$$
Why is this defined as the return? Is there anything useful about this?
It seems like it's more useful to define the return as $$G_t=R_0+ \dots+R_t,$$ because your "return", so to speak, is the "profit from investment" so it seems like your return will be your accumulated reward from taking actions up to that point.

It wouldn't make sense to define the return as you propose, from time 0 to $t$. Once we are in a state at time $t$ we don't care what the returns have been, rather what they will be in the future, thus returns are defined as the total sum of discounted rewards from the current time step onwards. This allows the agent to make decisions about which actions to take based on how valuable taking said action is in the current state at time $t$ -- clearly the rewards previous to this have no effect upon that.


Can programs like AlphaGo be said to be means of dealing with computational intractability? I mean this in the sense that Go is unsolvable but AlphaGo seems able to make choices that are consistently more optimal than a human player's choices.

It is my understanding that Game Theory turned out to have limited applications in real world scenarios because of the profound complexity of such scenarios and degree of hidden information.  Is it fair to say that there is now a method for dealing with this?

I fully understand that Go is a game of complete information, which has a very specific meaning, but it occurs to me that the inability to generate a complete game tree (computational intractability) could be seen as form of incomplete information, even if it is not traditionally thought of in those terms.

I should probably note that my perspective is one of a "serious" game designer, where complexity serves the same function as chance and hidden information, which is to say as a balancing factor that "levels the playing field".

I think that the technique AlphaGo used to solve the computational intractability problem of the search space are not new, it uses the Monte Carlo Tree Search. The real innovation in AlphaGo was to figure out how to compute the evaluation function of a move, that was the really tricky part. For this they used combinations Deep and Reinforcement learning techniques.


Chatbots triggering emotions I’m a researcher and I’m currently conducting a research project. I will conduct a study where I would like to trigger different emotions using chatbots on a smartphone (e.g. on Facebook Messenger).

Are there any existing chatbots which are able to trigger different emotions intentionally (also negative ones)?

Emotions can of course be triggered by lots of different things. I think the most rich source could well be socialbots like Mitsuku.com and Zo.ai -- Steve Worswick is the owner of Mitsuku and may be interested in helping you by doing (appropriately filtered) chat log queries. You can get him on Twitter at @Mitsuku.


GAN model predictions before training is predictable I have a dataset of 3000 8x8 images, and I would like to train a GAN for an image generation purpose.
I am planning to start with a simple GAN model and see if it overfits. Before training, I try to do a comparison of the discriminator model prediction using real image input against the whole GAN model prediction using random seed input. My thought process is that since this model is not trained yet, the output for real images and fake images by the discriminator should not be predictable.
However, the discriminator model prediction using real image input always returns a value very close to 1.0, and the whole GAN model prediction using random seed input always returns a value near 0.5 with a small deviation. I suspect that during training, the model would simply pull the 0.5 value near 0.0 and would never actually learn from the dataset.
I try to increase the training parameters and different initializers, but the output is still the same.
By ruling out the possibility a bad dataset, what could be the reason for this situation?
This is some sneak peek of the generator and discriminator model building: https://pastebin.com/ehMDP7k6

I took a look at your model. It seems you have incorrect architecture. The Conv2D layers in your D should have following params: (n_filters, kernel=3, padding='same'), where n_filters is the number of filters and it usually should be doubled as per DCGAN architecture. You can also use strides but since your images are small it won't make any sense.
The D network also should not include UpSampling2D layer. This layer can be used in G network instead of Conv2DTranspose.
Your images should be normalized to the range of [-1, 1] and the activation function of G should be tanh.
You also included m.add(keras.layers.Reshape((8, 8, 3))) to your G network which is the final size of your data. Since you have the final size in the first layers you don't have to include upsampling Conv2DTranspose layers.
Your final model should look like
def make_discriminator_model():
    model = tf.keras.Sequential()
    model.add(layers.Conv2D(8, kernel=3, strides=1, padding='same',
                                     input_shape=[8, 8, 1]))
    model.add(layers.LeakyReLU())
    model.add(layers.Dropout(0.3))

    model.add(layers.Conv2D(16, kernel=3, strides=1, padding='same'))
    model.add(layers.LeakyReLU())
    model.add(layers.Dropout(0.3))

    model.add(layers.Flatten())
    model.add(layers.Dense(1))

    return model

def make_generator_model():
    model = tf.keras.Sequential()
    model.add(layers.Dense(8*8*16, use_bias=False, input_shape=(100,)))
    model.add(layers.BatchNormalization())
    model.add(layers.LeakyReLU())

    model.add(layers.Reshape((8, 8, 16)))

    model.add(layers.Conv2D(8, kernel=3, strides=1, padding='same'))
    model.add(layers.BatchNormalization())
    model.add(layers.LeakyReLU())

    model.add(layers.Conv2D(1, kernel=3, strides=1, padding='same', activation='tanh'))

    return model

Note that I did not test the model, so you should adapt it to the shape of your data.
I encourage you to follow this tutorial.
Update regarding overfitting
GANs do not have overfitting problem in the classical sense. Instead, vanilla GAN have other problems like vanishing gradient. It happens when the D becomes overconfident regarding fake samples. In that case, D stops providing useful information to the G, since its error becomes 0. To find out if training fell into this problem, you should plot D and G loss. It will look as follows:

Another problem is the mode collapse. In that case, the G tricks the D by producing only one type of samples which looks realistic but G won't represent the real data distribution. Therefore, the generated samples will be homogeneous.
For more information, see Improved Techniques for Training GANs and Wasserstein GAN.


Game AI - Fast python OCR or cropped image input I'm developing a Game AI which tries to master racing simulations. I already trained a CNN (alexnet) on ingame footage of me playing the game and the pressed keys as the target. As the CNN is only making predictions on a frame-to-frame basis, and I resized the image input to 160x120 due to GPU memory limitations, it cannot read the speedometer, thus seems not to have a feeling for its current velocity. I thought of different ways to fix this issue:

Crop the captured image down to the size of the speedometer, which displays the current speed in mph, and feed the low resolution game image, as well as the relatively high-res image (70x30) of the current speed into the neural network, which makes predictions based on the two images.

As I don't know whether alexnet can serve as an OCR as well, my second thought was to use an existing one (like tesseract-ocr/PyTesser) on the cropped image and feed its output to the fully connected layer.

I already tried to implement an optical-flow algorithm, but sadly, non of the python ones seems to output good real-time results. I wonder whether I can input the current frame as well as the last one, and let alexnet figure out the movement.

As the processing has to happen in real time, and the only performance reviews of pytesser I found reported a processing time of ~100ms (never tested that). My Question is, what method would work best.
Thanks!
Edit: Optical flow would have the advantage of the AI knowing in which direction other cars are moving as well.

Option 1 would be a very interesting one from a research perspective. I cannot imagine that CNNs have to capacity at the moment, to learn the concept of numbers and apply them in a useful way. If it was an analog speedometer, things would be different. But it would be really interesting to try it and see what you can achieve with this approach. I haven't read any research papers yet, where such a challenge would have been mastered.

If you are less concerned with research but rather get this project to work, I would propose an approach similar to option 2. The OCR tools you mentioned were designed to identify symbols, even if distorted or otherwise hard to read. In your case, the numbers will always look the same and are most likely always in fixed positions. Therefore, using fancy OCR algorithms or neural nets are overkill for the problem at hand. You can write a simple algorithm that crops the speedometer, searches for the 10 possible patterns (0 to 9) with 10 specific kernels and calculate the speed from the result. This can be implemented efficiently without the need to train a CNN or some other complex algorithm.

Method for implementation:

To answer the first question in the comments. The algorithm for this is really easy. Your image will be represented by a 2D array (or maybe a 3D array if you are working with colors). All you need to do is find out, what your numbers look like and store the appropriate array. You should end up with 10 different arrays for 10 different numbers.

To find out if a number is in your current picture, you only have to check if the array for this number is a subset of the array of your current picture. The position, where the arrays are matching, will also indicate the position of the number. You do this for all 10 numbers and can calculate the actual value displayed on the speedometer afterwards from the results.

This is very easy if the numbers always look exactly identical. In this case you can simply look for an exact match. If there are slight variations, you can look for parts of the image array that is similar to your template arrays. Just sum the absolute values of the diffs of each pixel value and if it is below a certain threshold, you have your match. That's all there is to it.

In python I like working with PIL for this kind of image manipulation, but there are other frameworks out there as well.


Why spend so much time and money to build AIs to play games? I was reading about John McCarthy and his orthodox vision of Artificial Intelligence. To me, it seems like he was not very much in favour of resources (like time and money) being used to make AIs play games like Chess. Instead, he wanted more to focus on passing the Turing test and AIs imitating human behavior.
I have also read many articles about major companies, like IBM, Google, etc., spending millions of dollars in making AIs play games, like Chess, Go, etc.
To what extent is this justified?

In the book Artificial Intelligence: A Modern Approach (section 5.7, p. 185), Russell and Norvig write

In 1965, the Russian mathematician Alexander Kronrod called chess "the Drosophila of artificial intelligence." John McCarthy disagrees: whereas geneticists use fruit flies to make discoveries that apply to biology more broadly, AI has used chess to do the equivalent of breeding very fast fruit flies. Perhaps a better analogy is that chess is to AI as Grand Prix motor racing is to the car industry: state-of-the-art game programs are blindingly fast, highly optimized machines that incorporate the latest engineering advances, but they aren't much use for doing the shopping or driving off-road. Nonetheless, racing and game-playing generate excitement and a steady stream of innovations that have been adopted by the wider
community

So, although these games (like Chess, Go, and Bridge) may not apparently be useful/beneficial to many people, the AI programs developed to play them have introduced/included concepts/techniques, like null move heuristics, futility pruning, combinatorial game theory, finessing and squeezing or meta-reasoning, which can potentially be useful to a wider spectrum of Computer Science (and not just Artificial Intelligence).
You can see it similar to space missions of NASA, ISRO, JAXA and other space agencies. All these missions don't seem to have a direct benefit to citizens, but have many indirect benefits. They pave the way for technological innovations (GPS, 3D printing, car crash technology, clean energy, LED), the creation of jobs, etc. Advance storms, hurricane detection is the output of space exploration, which has saved millions of lives worldwide.
AI in games has not just helped to develop the software but hardware also. Many innovations have been seen to produce highly optimised and powerful hardware.


What kind of hardware and software are required to build an AI system? What kind of hardware (processor, memory, graphics card, etc.) and software are required to build an AI system? How do we go about linking each and every one of them to their proper place and function?

What is needed besides the main hardware or software of a computer that will be used to make an AI system?

Many different components may go into building an AI system. AI is a broad topic encompassing, machine learning, statistical learning so an AI system could involve anything from a raspberry pi to a spoon. Your title refers specifically to processors of an AI system. The mathematics behind AI can be run on any processor. GPU, CPU, FPGA, and new Neural network chips. A neural network can be run on a Window PC a linux PC AI packages are not OS dependent. I can train a model on my laptop now.

2019-11-14 20:10:57.503229:
Epoch 1/5
60000/60000 [==============================] - 8s 137us/sample - loss: 0.2925 - acc: 0.9141
Epoch 2/5
60000/60000 [==============================] - 8s 140us/sample - loss: 0.1426 - acc: 0.9574
Epoch 3/5
60000/60000 [==============================] - 9s 142us/sample - loss: 0.1059 - acc: 0.9674
Epoch 4/5
60000/60000 [==============================] - 8s 135us/sample - loss: 0.0862 - acc: 0.9732
Epoch 5/5
60000/60000 [==============================] - 7s 112us/sample - loss: 0.0744 - acc: 0.9763

Just like any computer application an AI needs a processor and memory to store information. Beyond that it is difficult what specifically you are looking for.

A configurable cloud-scale DNN processor for real-time AI


What is wrong with this CNN network, why are there hot pixels? I'm building a CNN decoder, which mirrors (in reverse) the VGG network structure from Conv-4-1 layer.

The net seems to be working fine, however, the output looks broken. Please note that the colour distortion is fine, it's the the [255/0 RGB pixels] e.g. green that I'm worrying about.

I tried to overfit a single image, but even then I get these hot pixels. Does anyone know why they appear?

My net:

    activation = 'elu'

    input_ = Input((None, None, 512))
    x = Conv2D(filters=256, kernel_size=self.kernel_size, padding='same', bias_initializer='zeros', activation=activation)(input_)

    x = UpSampling2D()(x)
    for _ in range(3):
        x = Conv2D(filters=256, kernel_size=self.kernel_size, padding='same', activation=activation)(x)
    x = Conv2D(filters=128, kernel_size=self.kernel_size, padding='same', activation=activation)(x)

    x = UpSampling2D()(x)
    x = Conv2D(filters=128, kernel_size=self.kernel_size, padding='same', activation=activation)(x)
    x = Conv2D(filters=64, kernel_size=self.kernel_size, padding='same', activation=activation)(x)

    x = UpSampling2D()(x)
    x = Conv2D(filters=64, kernel_size=self.kernel_size, padding='same', activation=activation)(x)
    x = Conv2D(filters=3, kernel_size=self.kernel_size, padding='same')(x)

    model = Model(inputs=input_, outputs=x)

I've seen this too many times - it's not a problem with your network, it's a problem with matplotlib and how it displays the image. You are probably trying to display a float with range $<0, 255>$. When matplotlib sees float type as input, it assumes a range of $<0, 1>$, and thresholds everything outside of that range, and the results you can see.


CNN for detecting not just the nature of the object, but position within image as well Consider a typical convolutional neural network like this example that recognizes 10 different kinds of objects from the CIFAR-10 dataset:

https://github.com/tflearn/tflearn/blob/master/examples/images/convnet_cifar10.py

""" Convolutional network applied to CIFAR-10 dataset classification task.

References:
    Learning Multiple Layers of Features from Tiny Images, A. Krizhevsky, 2009.

Links:
    [CIFAR-10 Dataset](https://www.cs.toronto.edu/~kriz/cifar.html)

"""
from __future__ import division, print_function, absolute_import

import tflearn
from tflearn.data_utils import shuffle, to_categorical
from tflearn.layers.core import input_data, dropout, fully_connected
from tflearn.layers.conv import conv_2d, max_pool_2d
from tflearn.layers.estimator import regression
from tflearn.data_preprocessing import ImagePreprocessing
from tflearn.data_augmentation import ImageAugmentation

# Data loading and preprocessing
from tflearn.datasets import cifar10
(X, Y), (X_test, Y_test) = cifar10.load_data()
X, Y = shuffle(X, Y)
Y = to_categorical(Y, 10)
Y_test = to_categorical(Y_test, 10)

# Real-time data preprocessing
img_prep = ImagePreprocessing()
img_prep.add_featurewise_zero_center()
img_prep.add_featurewise_stdnorm()

# Real-time data augmentation
img_aug = ImageAugmentation()
img_aug.add_random_flip_leftright()
img_aug.add_random_rotation(max_angle=25.)

# Convolutional network building
network = input_data(shape=[None, 32, 32, 3],
                     data_preprocessing=img_prep,
                     data_augmentation=img_aug)
network = conv_2d(network, 32, 3, activation='relu')
network = max_pool_2d(network, 2)
network = conv_2d(network, 64, 3, activation='relu')
network = conv_2d(network, 64, 3, activation='relu')
network = max_pool_2d(network, 2)
network = fully_connected(network, 512, activation='relu')
network = dropout(network, 0.5)
network = fully_connected(network, 10, activation='softmax')
network = regression(network, optimizer='adam',
                     loss='categorical_crossentropy',
                     learning_rate=0.001)

# Train using classifier
model = tflearn.DNN(network, tensorboard_verbose=0)
model.fit(X, Y, n_epoch=50, shuffle=True, validation_set=(X_test, Y_test),
          show_metric=True, batch_size=96, run_id='cifar10_cnn')

It's a CNN with several layers, ending with 10 outputs, one for each type of object recognized.

But now think of a slightly different problem: Let's say I only want to recognize one type of object, but also detect its position within the image frame. Let's say I want to distinguish between:

object is in center
object is left of center
object is right of center
no recognizable object

Assume I build a CNN exactly like the one in the CIFAR-10 example, but only with 3 outputs:

center
left
right

And of course, if none of the outputs fires, then there is no recognizable object.

Assume I have a large training corpus of images, with the same kind of object in many different positions within the image, the set is grouped and annotated properly, and I train the CNN using the usual methods.

Should I expect the CNN to just "magically" work? Or are there different kinds of architectures required to deal with object position? If so, what are those architectures?

You could use another type of CNN that instead of classification is performing regression so it will also give you as output the position(it's not really like that but this is the core idea) .
Some algorithms are SSD or YOLO.


Why is the validation performance better than the training performance? I am training a classifier to identify 24 hand signs of American Sign Language. I created a custom dataset by recording videos in different backgrounds for each of the signs and later converted the videos into images. Each sign has 3000 images, that were randomly selected to generate a training dataset with 2400 images/sign and validation dataset with the remaining 600 images/sign.

Total number of images in entire dataset: 3000 * 24 = 72000
Training dataset: 2400 * 24 = 57600
Validation dataset: 600 * 24 = 14400
Image dimension (Width x Height): 1280 x 720 pixels

The CNN architecture used for training

model = Sequential([
    Conv2D(32, (3, 3), activation='relu', input_shape=(IMG_HEIGHT, IMG_WIDTH ,3)),
    MaxPooling2D(pool_size=(2,2)),
    Dropout(0.25),

    Conv2D(32, (3, 3), activation='relu'),
    MaxPooling2D(pool_size=(2,2)),
    Dropout(0.25),

    Conv2D(64, (3, 3), activation='relu'),
    MaxPooling2D(pool_size=(2,2)),
    Dropout(0.25),

    Conv2D(64, (3, 3), activation='relu'),
    MaxPooling2D(pool_size=(2,2)),
    Dropout(0.25),

    Flatten(),
    Dense(128, activation='relu'),
    Dropout(0.25),

    Dense(NUM_CLASSES, activation='softmax')
])

Training parameters:

IMG_HEIGHT = 224
IMG_WIDTH = 224
BATCH_SIZE = 32
NUM_CLASSES = 24
train_datagen = ImageDataGenerator(rescale = 1./255,
                                   width_shift_range=0.1,
                                   height_shift_range=0.1,
                                   zoom_range=0.1,
                                   fill_mode='constant')
EPOCHS = 20
STEPS_PER_EPOCH = TRAIN_TOTAL // BATCH_SIZE
VALIDATION_STEPS = VALIDATION_TOTAL // BATCH_SIZE

callbacks_list = [
    tf.keras.callbacks.EarlyStopping(monitor = 'accuracy',
                                     min_delta = 0.005,
                                     patience = 3),
    tf.keras.callbacks.ModelCheckpoint(filepath = 'D:\\Models\\HSRS_ThesisDataset_5Mar_1330.h5',
                                       monitor= 'val_loss',
                                       save_best_only = True)
]

optimizer = 'adam'

The model accuracy and model loss graph is shown in the figure below:

The results obtained at the end of the training are

Train acc: 0.8000121
Val acc: 0.914441

I read this article explaining why the validation loss is lower than the training loss I want to know:

Is it because of the smaller dataset and random shuffling of the images?
Is there any way to improve the condition without changing the dataset?
Will this have a very detrimental effect on the model performance in real test cases? If not, can I just focus on improving the training accuracy of the overall model?

Assuming you pass through the entire validation dataset, this can't be due to shuffling since you still compute the loss/accuracy over the entire dataset, so order does not really matter here. It is more likely that you have a significantly smaller or less representative validation dataset, e.g., distribution of the validation dataset can be skewed towards classes where your model performs better.
What do you mean exactly by improving the situation? Having a better validation accuracy is not necessarily bad. In any case, if you decrease the effect of regularization, e.g., lowering weight decay, training accuracy might go up but your model might generalize worse, i.e.,  you might get a lower validation accuracy.
No, the goal of training is never to maximize training accuracy. You can trivially do so by just memorizing the training dataset. In short, the goal of training is to get good generalization and as long as you get a satisfactory validation accuracy, it is likely that this has happened to some degree (assuming you have a good validation dataset of course).


How can I have a computer learn the equation with known dependent variables? I'm trying to design an orbital rendezvous program for Kerbal Space Program.  I'm focusing on when to launch my spacecraft so that I can end up in the general vicinity of the target.  If I can control the ascent profile, the remaining dependent variables are the ship's twr and the target's altitude.  I want to try a computer learning solution.

What is the best way to formulate the problem of learning the time to launch based on some twr?

How can I make an algorithm to compute the general equation of launch time to a altitude based on my ability to accelerate?  What type of learning problem could this be classified as?  What are some approaches to solve problems with known dependent variables?

This may be an obvious question, I kind of expect the answer is regression?  But it seemed a general enough question to be sure about to get a solid foothold in computer learning with this type of problem, which seems to come up a lot.

You might want to try symbolic regression. This is a machine learning approach that tries to generate an equation with arbitrary form that best fits a set of examples. I've used it before when working with physicists who wanted an equation in terms of known numeric constants and specific variables, but didn't expect the equation to be linear or one of the other shapes for which we have standard regression models.

Symbolic regression often (usually?) uses genetic programming for the underlying optimization. A decent tool is Eureka. It used to be free, but I think they want a payment now. You can also do it yourself from any genetic programming toolkit with a little know-how.


What is the effect of parallel environments in reinforcement learning? Do parallel environments improve the agent's ability to learn or does it not really make a difference? Specifically, I am using PPO, but I think this applies across the board to other algorithms too.

Do parallel environments improve the agent's ability to learn or does it not really make a difference?

Yes they can make a difference. There are two ways improvement is seen:

Collecting data from multiple trajectories at once reduces correlation in the dataset. This improves convergence for online learning systems like neural networks, which work best with i.i.d. data.

Data collection is faster overall, which improves clock time to obtain the same result. This may make better use of other resources too.

Of the two, the first improvement is important for stability, although it can be emulated by running multiple episodes - or restarting from multiple starting points - between batch learning updates.

Specifically, I am using PPO, but I think this applies across the board to other algorithms too.

It does apply to PPO, but the first improvement does not apply across the board. These things need to be true for environment run in paralell to be help with stability:

Using an on-policy method, or where experience replay is not an option.

Using a function approximator for policy and/or value function.

A lot of policy gradient methods match this, including PPO, A3C, REINFORCE. However, for an off-policy method like DQN, the main benefit will be faster data collection.
These effects are discussed in sections 1 and 4 of the paper Asynchronous Methods for Deep Reinforcement Learning which introduced A3C - thanks to DeepQZero for that reference.


To what extent can quantum computers help to develop Artificial Intelligence? What aspects of quantum computers, if any, can help to further develop Artificial Intelligence?

Quantum computers are super awesome at matrix multiplication, with some limitations. Quantum superposition allows each bit to be in a lot more states than just zero or one, and quantum gates can fiddle those bits in many different ways. Because of that, a quantum computer can process a lot of information at once for certain applications.

One of those applications is the Fourier transform, which is useful in a lot of problems, like signal analysis and array processing. There's also Grover's quantum search algorithm, which finds the single value for which a given function returns something different. If an AI problem can be expressed in a mathematical form amenable to quantum computing, it can receive great speedups. Sufficient speedups could transform an AI idea from "theoretically interesting but insanely slow" to "quite practical once we get a good handle on quantum computing."


How is Monte Carlo different from model-based methods? I was going through an article where it is mentioned:

  The Monte-Carlo methods require only knowledge base (history/past experiences)—sample sequences of (states, actions and rewards) from the interaction with the environment, and no actual model of the environment.

Aren't the model-based method dependent on past sequences? How is Monte Carlo is different than?

Model-based methods (such as value or policy iteration) use a model of the environment, which is usually represented as a Markov decision process. More specifically, the model consists of the transition and reward functions of the Markov decision process, which should represent the dynamics of the environment. For example, in policy iteration, the rewards (used to estimate the policy or value functions) are not the result of the interaction with the environment but given by the MDP (the model of the environment), so the decisions are made according to the reward function (and the transition function) of the MDP that represents the dynamics of the environment. Model-based methods are not (usually) dependent on past actions. For example, policy iteration converges to the optimal policy independently of the initial values of the states, the initial policy or the order of iteration through the states.

Monte Carlo methods do not use such a model (the MDP), even though the assumption that the environment can be represented as an MDP is (often implicitly) made (and the MDP might actually be available). In the case of Monte Carlo methods, all estimates are solely based on the interaction with the environment. In general, Monte Carlo methods are based on sampling (or random) operations. In the case of reinforcement learning, they sample the environment. The samples are the rewards that are obtained when certain actions are taken from certain states.


What were the first areas of research and what were some early successes? What were the first areas of research into Artificial Intelligence and what were some early successes?  More recently we've had:

Beating a human at the game of chess
Convincing a human that a person was conversing with them (passing the Turing test)
Beating a human at Jeopardy game show
Beating a human at the game of go.

Were there milestones that were considered major in the field before the 1990s?

What might be classed as AI has of course changed over the years, but landmarks and research breakthroughs include:

Babbagge's Difference Engine (~1823) for tabulating/interpolating polynomials.
Frank Rosenblatt's 1957 invention of the Perceptron.
John McCarthy's invention of Lisp in the late 1950s.
Arthur Samuel's 1959 checkers player, which famously improved by playing against itself (it would have been nice if that had destroyed the myth about a program only being as 'intelligent' as its creator).
Newell and Simon's 1959 General Problem Solver applied Means-Ends analysis to solve a range of problems expressed as Horn clauses.
Davis, Putnam et al: 1962 invention of the DPLL algorithm which still forms the core of modern SAT-based theorem provers.
Lawrence Fogel et al: 1966 book Artificial Intelligence through Simulated Evolution.
Rechenberg and Schwefel: 1960s development of Evolutionsstrategie - an Evolutionary Computation approach using mutation and a form of Darwinian 'survival of the fittest'.
Lotfi Zadeh's 1965 invention of Fuzzy Logic.
John Holland's 1975 book "Adaptation in Natural and Artificial Systems" which introduced Genetic Algorithms.
The 1980 Hearsay II Blackboard Architecture by Hayes-Roth et al.
The 1980s invention of the backpropagation algorithm for Mutlilayer Perceptrons by Rumelhart, Hinton et al.


How exactly does Angelina design games? In the recent festival of science, there was a talk given by researcher Mike Cook about:

  ANGELINA, an AI game designer that has invented game mechanics, made games about news stories, and was the first AI to enter a game jam.

So the aim of Angelina AI is basically to design videogames.

Briefly, how exactly does Angelina design the new games? How does it work behind the scenes?

In any case, it appears that much of his past work on Github involves procedural generation. Which is AI... ish. Unless there's more to it, which we can't see because half the site is down.
This paper appears to offer an analysis of combining procedural generation with game AI.
From the abstract:

Populated and immersive game contexts require large numbers of minor,
background characters to fill out the virtual environment. To limit
game AI development effort, however, such characters are typically
represented by very simplistic AI with either little difference
between characters or only highly formulaic variations. Here we
describe a complete workflow and framework for easily designing,
generating and incorporating multiple, interesting game AIs. Our
approach uses high-level, visual Statechart models to represent
behaviour in a modular form; this allows for not only simplistic,
parameterbased variation in AI design, but also permits more complex
structure-based approaches. We demonstrate our technique by applying
it to the task of generating a large number of individual AIs for
computer-controlled squirrels within the Mammoth 1 framework for
game research. Rapid development and easy deployment of AIs allow us
to create a wide variety of interesting AIs, greatly improving the
sense of immersion in a virtual environment.

Here's an article from 2015 on AI and procedural generation, which discusses Angelina at length. This article links to a more in-depth article from 2013.
Here's an excerpt:

Cook gave ANGELINA the ability to learn about people so that it could
make games based on current events. Then Cook gave ANGELINA memory -
that is, the ability to keep track of the people it had learned about.
The memory's not a big deal, even though it led to a number of
philosophical disagreements around Cook's desk. ANGELINA's memory is
actually just a text file where it stores the names of all the people
it's heard of, alongside a number: a measure of its opinion of them
based on the things it's learned from internet chatter. It liked
Al-Assad more than May. It liked everyone more than May.


Are strong Chess AI's local on mobile devices? Just wondering about the architecture of strong Chess AI in a mobile, because networking is generally assumed by mobile developers, but not guaranteed.

Yes. In fact, I'm the developer of some mobile chess apps. [Disclaimer]

I would argue my chess engine products for the iOS (http://www.smallchess.com) are powerful and strong enough to beat any chess grandmaster. My apps run without network connection,


When is the Markov decision process not adequate for goal-directed learning tasks? In the book Reinforcement Learning: An Introduction (Sutton and Barto, 2018). The authors ask

  Exercise 3.2:  Is the MDP framework adequate to usefully represent all goal-directed learning tasks? Can you think of any clear
  exceptions?

I thought maybe a card game would be an example if the state does not contain any pieces of information on previously played cards. But that would mean that the chosen state leads to a system that is not fully observable. Hence, if I track all cards and append it to the state (state vector with changing dimension) the problem should have the Markov Property (no information on the past states is needed). This would not be possible if the state is postulated as invariant in MDP.

If the previous procedure is allowed, then it seems to me that there are no examples where the MDP is not appropriate.

I would be glad if someone could say if my reasoning is right or wrong. What would be an appropriate answer to this question?

Background
The Markov Decision Process is an extension of Andrey Markov's action sequence that visualize action-result sequence possibilities as a directed acyclic graph.  One path through the acyclic graph, if it satisfies the Markov Property is called a Markov Chain.
The Markov Property requires that the probability distribution of future states at any point within the acyclic graph be evaluated solely on the basis of the present state.
Markov Chains are thus a stochastic model theoretically representing one of the set of possible paths.  And the action-result sequence is a list of state transitions corresponding to actions chosen solely by each action's previous state and the expectations that the expected subsequent state will most probably lead to the desired outcome.
Andrey Markov based his work on Gustav Kirchhoff's work on spanning trees, which is based on Euler's initial directed graph work.
The Exercise
Exercise 3.2 was given with two parts.

Is the MDP framework adequate to usefully represent all goal-directed learning tasks?
Can you think of any clear exceptions?

The first question is subjective in that it inquires about usefulness but does not define what it means.  If "useful" means the MDP will improve the chances of achieving a goal over a random selection of action at each state, then except in no win scenarios or the most contrived case where all actions have equal distribution of probable results, then the MDP is useful.
If "useful" means optimal, then there are other approaches, with additional complexity and requiring additional computing resources that will improve odds of goal achievement.  These other approaches overcome one or more of the limitations of pure MDP.
Advancements and Alternatives
Advancements made to MDP and alternatives to MDP, which number in the hundreds, include these.

Logical detection of the infeasibility of goal achievement (no win scenario)
Calculation of probabilities when only partial information is available about the current state
Invocation of the decision at any point (continuous MDP used in real time systems)
Probabilities are not known and must be learned from past experience where simple Q-learning is employed
Past experience is used by statistically relating action-state details to generalizations derived from past action-result sequences or such information acquired or shared
The action-state decisions, made within the context of an unknown system of changing or not reliably applied rules, can be used to tune a set of fuzzy rules in a fuzzy logic container and utilize fuzzy inference in the decisions
Bluff and fraud detection

Card Games
Game play for a typical card game could make use of MDP, so MDP would be strictly useful, however not optimal.  Some of the above decisioning features would be more optimal, particularly those that deals with unknowns and employ rules, since the card game has them.
Random or Decoupled
Two obvious cases are (a) a truly randomized action-result world where goal achievement has equal probability no matter the sequence of moves or (b) a scenario where goal achievement is entirely decoupled from actions the actor can take.  In those cases, nothing will be useful with regard to the particular objective chosen.
Challenge
The way to best learn from the exercise, though, is to find a scenario where MDP would be useless and one of the above listed Advancements and Alternatives would be required rather than simply preferred.  If you look at the list, there will be some cases that will eventually come to mind.  I suggest you think it through, since the goal is to learn from the book.


How can I detect the frame from video streaming that contains a graffiti on city wall? I am working on a graffiti detection project. I need to analyze data stream from a camera mounted sideways on a vehicle to identify graffiti on city walls and notify authorities with the single best capture of graffiti and its geolocation, etc.

I am trying to use a ResNet50 model pre-trained on ImageNet using transfer learning for my graffiti image dataset. The classification will be done on an edge device as network connectivity may not be reliable.

Suppose I have a series of frames that have been detected to contain graffiti, as the vehicle goes past it, but I only need to report one image (so not all frames containing graffiti in the series). How can I do that?

Ideally, I would like to report the frame where the camera is perpendicular to the wall. Why perpendicular? I think that images containing the graffiti when the camera is perpendicular to the wall will more clearly show the graffiti.

ResNet is an architecture for object recognition and you may use it to do your classification task.  Fast RCNN may improve your results but is a more difficult architecture to implement.  If you want to go in this direction the best place to start is the arxiv paper of the Fast R-CNN (arxiv.org/abs/1504.08083).  If I am not wrong, there is an implementation of fast-rcnn in pytorch if you really want that.

You seem to be new to deep learning.  I would strongly recommend starting simple. ResNet-50 will probably be more than enough for your application. Moreover, I suggest using a lib like fast.ai.


Can imbalance data create overfitting? I am doing human activity recognition project. I have total of 12 classes. The class distribution look like this:

$\color{red}{If \ you \ watch \ carefully, you \ can \ see \ that \ I \ have \ no \ data \ points \ for \ class \ 11 \ and \ class \ 8.}$ Also, the dataset is highly imbalanced. So, I took minimum data points (in this case 2028) for all of the classes. Now my balanced data look like this:

After doing this it looks like a balance data. $\color{red}{But \ still, \ I \ think \ it \ not, \ because \ I \ have \ zero \ datapoints \ for \ class \ 11 \ and \ class \ 8}$. In my opinion the classes are still imbalance.
I am using CNN model to solve this activity project. My model summary is following:

The main problem is, my model starts overfitting heavily when I train it.

Is it due to my imbalance data( class 8 and 11 has zero data points) or something else?
$\textbf{Hyperperameter:}$
$\textbf{features:}$ X, Y, Z of mobile accelerometer
$\textbf{frame size:}$ 80
$\textbf{optimizer:}$ Adam, $\textbf{Learning rate:}$ 0.001
$\textbf{Loss:}$ Sparse categorical cross-entropy

Your zero-classes just do not exist for the model. There is no information about them in training and test sets. I think the reason for overfitting is that your have very small size of the training set comparing to the number of parameters in your model. If it is easier for a model to remember all training entries - it will just do that. You need to prune your model to force it to make some generalizations about the data.


Which field to study to learn & create a.i generated simulations? I wasn't sure how to title this question so pardon me please.

You may have seen at least one video of those "INSANE A.I created simulation of {X} doing {Y & Z} like the following ones:

A.I learns how to play Mario
A.I swaps faces of {insert celebrity} in this video after 16hrs.
etc...

I want to know what I have to learn to be able to create for example a program that takes xyz-K images of a person as training data and changes it with another person's face in a video.

Or create a program that on a basic level creates a simulation of 2 objects orbiting /attracting each other /colliding like this:

What field/topic is that?
 I suspect deep learning but I'm not sure. I'm currently learning machine learning with Python.

I'm struggling because linear regression & finances /stock value prediction is really not interesting compared to teaching objects in games to do archive something or create a program that tries to read characters from images.

You need to define "simulation" more specific. Playing Mario, Swapping face on image/video, or generating simulation of objects that are orbiting use different techniques.

Playing Mario or "AI that playing game": the AI agent trained on available environment (Mario game, so the environment is not generated) and learn the best sequential actions to achieve the goal. It runs the game thousand times, when it did a wrong action then it gets "penalties" that improve its knowledge. The algorithm that can be used is Reinforcement Learning, but some earlier paper use Genetic Algorithm to generate the best action
Face swap: It's close to computer vision area, some methods that I know use Style Transfer principle (Convolutional Neural Network) to make transformation of face of one image to another image. You can read the basic of style transfer here.
Generating physical movement: I don't know too much about this topic but I know there are some papers talk about this, Fluid Net from Google workers and this paper from TU Munchen. At a glance they also use CNN to improve the result but the main simulation came from Euler Fluid Equation. So if you need to generate object that orbiting, I think you need to find equations that models that movement.

Hope it helps!


How can blackjack be formulated as a Markov decision process? I am reading sutton barton's reinforcement learning textbook and have come across the finite Markov decision process (MDP) example of the blackjack game (Example 5.1).

Isn't the environment constantly changing in this game? How would the transition probabilities be fixed in such an environment, when both you and the dealer draw cards?

Isn't the environment constantly changing in this game?

The current state of the agent and the environment is constantly changing as you play, but not necessarily the transition probabilities. For simplicity, you may assume that the transition probabilities do not change (e.g. if the dealer and the deck are the same every time you play).

  How would the transition probabilities be fixed in such an environment, when both you and the dealer draw cards?

The actions of the dealer will be incorporated into the transition probabilities of the environment. Whenever the player (or RL agent) takes an action, then it will receive a reward, according to the reward function of the environment (the rules of the game), and the agent and the environment will be moved to the next state, according to the transition probabilities, which do not have to change for stochasticity to occur in the environment. In fact, these transition probabilities already incorporate this stochasticity.

Also, if even the environment was changing, you could still model your problem as an MDP, but the MDP would change accordingly.

The example 5.1 of the book (that you mention) actually explains in detail how to formulate this game as a finite MDP.


Can I develop a chatbot to carry on a natural conversation with a human using NLP and neural networks? I would like to develop a chatbot that is able to pass the Turing test, i.e. a chatbot that is able to carry on a natural conversation with a human.

Can natural language processing (NLP) be used to do that? What if I combine NLP with neural networks?

I would not recommend using neural networks and NLP together to create a system sufficiently capable of conversation/dialogue that it would pass that current crop of Turing-like tests.

Conversations follow certain rules and regularities (which we have only partially discovered so far), and training an ANN with dialogues in order to pick up those regularities is simply not feasible. In conversations you have a memory of what has been mentioned previously, you build up assumptions about the intentions of your dialogue partner, and keep track of the current topic and sub-topics. This is far too complex to be reduced to a machine learning approach.

As a starting point I would suggest looking at ELIZA, developed by Weizenbaum in the mid-1960s. There are plenty of implementations in various programming languages available. Use that as a starting point to extend the capabilities according to topics you want to talk about, and store in memory what the user has said before, trying to refer back to it, etc. This is a lot easier to do with 'symbolic' AI rather than subsymbolic processing.

A lot of current tech companies offer chatbot variants based on machine learning, but they rarely go beyond intent recognition or simple question-answer dialogues. For more sophisticated dialogues they are simply not suitable.

(Disclaimer: I work for a company producing conversational software)


How could Deep Blue possibly cheat? On Wikipedia we can read:

  Kasparov accused IBM of cheating and demanded a rematch. IBM refused and retired Deep Blue.

What was the accusation and how was Deep Blue allegedly able to cheat?

The allegation was based on the fact that Deep Blue made a choice that did not yield the immediate (or short term) benefit that was synonymous with systems back then (1997). Computational capability was significantly less powerful then, and Kasparov claimed that only a grand master would have made the decision that the system did - so the deep blue team cheated by having a human perform the move instead of the system.

He asked for a rematch, but IBM did not allow this, which only added to the suspicion.

This is a great article with deep analysis on the specific moves and circumstances - however suffice to say that Kasparov was trying to bait the system into making a decision for a weak pawn, but the system chose otherwise and instead put Kasparov into a compromised position:
https://en.chessbase.com/post/deep-blue-s-cheating-move


How to select the most appropriate set of actions for a given environment or task? Given a robot in a situation such as in a library reading a book.

Now I want to create a neural network that suggests an appropriate action in this situation. And, generally, ignore actions such as "get up and dance" and so on.

Since there are limitless actions a robot could do, I need to narrow it down to the ones in this situation. Using its vision system, the word "book" and book neurons should already be activated as well as "reading".

One idea I had was to create an adversarial network which generates words (sequences of letters) based on the situation such as "turn page", "read next line" and so on. And then have another neural network which translates these words into actions. (It would them simulate whether this was a good idea. If not it would somehow suppress the first word and try to generate a new word.)

Another example is the robot is in a maze and gets to a crossroads. The network would generate the word "turn left" and "turn right".

Another idea would be to have the actions be composed of a body part e.g. "eyes" and a movement such as "move left" and it would combine these to suggest actions.

Either way, it seems like I need a way to encode actions so that the robot doesn't consider every possible action in the universe.

Is there any research in this area or ideas on how to achieve this?

(I think this may be somewhat related to the task of "try to name as many animals as you can.")

One idea I had was to create an adversarial network which generates words

It may be better to use for e.g. numbers instead of words, then map these numbers to the language of your choice like English in your case. This mapping can be changed to other languages and you will have a robot that overcomes language barriers.


How are "lags" and "exogenous factors" accounted for in reinforcement learning? In reinforcement learning, the system sets some controllable variables, and then determines the quality of the result of the dependent variable(s); using that "quality" to update the algorithm.
In simple games, this works fine because for each setting there is a single result.
However, for the real world (e.g. an airflow system), the result takes some time to develop and there is no single precise "pair" result to the conditions set.  The flow change takes time and even oscillates a bit as flow stabilizes to a steady-state.
In practical systems, how is this "lag" accounted for?  How are the un-settled (false) results ignored?  How is this noise distinguished from exogenous factors (un-controlled system inputs e.g. an open window exposed to wind)?

One of Reinforcement Learning's core features is the ability to deal with delayed rewards/punishments - i.e. rewards that may occur as a consequence of a decision that occurred multiple time steps ago. That is because the value that it optimises is defined as a long-term sum of immediate rewards. This value is often called the return or utility and although it will take short term rewards into account, they do not have to be the dominant factors, and often are not.

All RL solvers are designed to solve this problem, it is core to the MDP formalism used in RL theory. You can decide to focus on immediate rewards or longer-term ones by adjusting using a discount factor (often represented as $\gamma$ in RL equations). Low discount factors will cause an agent to prefer short-term rewards, high discount factors will cause the agent to prefer for longer-term sums of reward. In continuous problems you must use $\gamma \lt 1$ or instead use average reward as the objective to be maximised.

One classic example of this kind of delay problem is the Mountain Car environment. Here, reward is only granted for reaching a certain point, and it is not possible to reach it by just taking the obvious action of moving directly towards the objective. Most RL algorithms can solve this problem, and get close to optimal solutions, purely from experiencing the final reward and associating it progressively with earlier and earlier decisions.

  In practical systems how is this "lag" accounted for?

As above, this is inherent to RL, as opposed to simplified versions such as Contextual Bandits which don't have time steps and evolving state.

In practice, it may take some experimentation to find the best RL algorithm and solve a problem efficiently. Environments with very large delays and lots of noise are harder to solve.

It is also really important to set the reward mechanism for your true goals, and not some intermediate ones. If your goal is a stable state, you need to reward that, and not necessarily maximum throughput. If both throughput and stability are important to the end result, you may need to reward both - e.g. reward throughput, but penalise short-term fluctuations.

  How are the un-settled (false) results ignored?

In general handling short-term vs long term effects of actions is known as the assignment problem.

This is most often solved statistically, over long term experience, as the agent associates not just immediate reward with an action, but the long term return after making each decision. Different RL solvers have different mechanisms for effectively moving this association back through time.

The key thing here is that an RL solver learns to maximise expected return or utility from any given state. Working with the expected value allows for smoothing out of variations, provided the agent gains enough experience to approach a statistical mean. Extremely rare and large variations can cause problems with this, and might need to be explicitly coded for as opposed to learned.

  How is this noise distinguished from exogenous factors (un-controlled system inputs e.g. an open window exposed to wind)?

This is trickier, and depends more critically on the problem.

A RL-based system that is exposed to these variations whilst learning, and some way to represent their effects in the state variables should learn to pick the best action when these external factors impact it.

RL systems based on maintaining a stable state can sometimes do well when they are moved out of that stable state. A good example here might be pole balancing, where the agent learns enough variations of position and speed that if something knocks the pole out of balance, it will still right itself even though the agent was not trained to react to such events directly.

However, in some cases, if knowledge is not available to the agent, or it has not been trained in that particular scenario, it will fail to generalise and may behave in counter-productive ways.

Either way, if you are concerned about specific scenarios, you need to include them in training, or at least test for them after training.


Is there a way to predict points on a map? I have a data set with historical information of some events (let's say event A and event B),these events describe the discovery of land mines, the coordinates of the event and the date of the event; is there a way I can use this historical information to predict points (coordinates) where event A or B could happen i.e. where might be still land mines that haven't been found?

Leaving aside the time aspect, you could do a cluster analysis on the event coordinates. If you use an algorithm that gives you a medoid (ie centre) of the clusters, you can then look at other points, and work out how close they are to the centres of the event clusters. It might be possible from this to predict which event could happen at those coordinates (which is the closest cluster medoid), and how likely it is (distance from the medoid).

This, however, depends very much on the shape of the data. If there is no discernible structure contained in it, then this will not work. But it is definitely worth trying.


Convolutional neural nets and reduction of the layers I have a very simple question about Conv nets. I understand the whole principle, but only one thing is not well explained on the Internet.

If I have a 16 channels image that goes on a convolutional layer, and the trainable filters are 3 7x7 filters, meaning that its output has 3 channels, how does the conv layer do to go from 16 to 3 channels? What mathematical operation is applied?

Thanks for any clarification on this.

The reason why you go from 16 to 3 channels is that by convention filters span the entire depth of the input. Therefore, your filters would actually be 7x7x16 in order to cover all channels of the input.

Detailed procedure

The output of the convolution automatically has a depth equal to the number of filters (so in your case this is 3) because you have an (m x k) filter matrix, where m is the number of filters and k is the number of elements in the unrolled filter (in your case, m = 3 and k = 7 x 7 x 16 = 784, so the filter matrix is 3 x 784).

The input is usually unrolled according to the im2col procedure, where each tile corresponding to a single filter location is stretched into a column equal to the unrolled filter size. This is repeated for each filter location, so you end up with very large matrix of size (k x n), where k is the same as k above in the filter matrix, and n depends on your padding and stride.

Multiplying the (m x k) filter matrix with the (k x n) input matrix gives you an (m x n) output matrix, where m is the number of filters.

Further reading

You can find some very nice visual explanations of the convolution procedure here and here.