Object IN/OUT counting using CNN+RNN I am building a video analytics program for counting moving things in a video. I am detecting bicycles and nothing else. I run object detection using the SSD mobile-net model in all the frames and store the bounding box coordinates (x,y,w,h) of each detection to a CSV file.

So for a video, I have a CSV file of one row each for a frame and each row has multiple detections of D1, D2, D3,.., Dn. Each detection has the bounding box coordinates as values. D1 is x,y,w,h.

Based on the x,y values of each detection, I am trying to find the direction of the bicycles and if the bicycle crosses the whole frame to do a UP/DOWN count.

How do I count/track(Don't want to use classic tracking algos) these bounding boxes moving in the video?

I see LSTM/RNN coming up in my search results when I search for video analytics. Being a noob, I am not able to find any tutorial that suits my needs.

I would like to check if my approach towards the problem is correct.

I don't want to use the classical tracking solutions for two reasons

I feel the tracking and counting conditions that I program in Python is always leaky/fails in certain conditions, hence I want to see how AI manages to count the objects.
The video stream I am using has heavy distortion on the objects that I track, hence the shape and size of the object changes drastically within 10s/20s of frames.

Any help or suggestion towards other better approaches is much appreciated.

Edit 1: The area of view under the camera is fixed. And we expect the bicycles to move from one entry side. Lets assume that the view and entry/exit is like shown in this video https://www.youtube.com/watch?v=tW7Pl3bSzR4

Assessing the Question

  Based on the x,y values of each detection, I am trying to find the direction of the bicycles and if the bicycle crosses the whole frame to do a UP/DOWN count.

It appears from the question that there is an interest in both counting bicycles and determining the direction of travel of each. from within the frames of a video stream or file. We can assume that each bicycle has at least one rider. It also appears that it is not a problem involving a fixed optical system positioned to point at a path that is tangential to the optical path, which would make the problem much easier, locking the approximate distance of the bicycle wheels to the optical system to near constant.

The use of the SSD mobile-net model seems reasonable as a starting point for developing expertise.

Starting With ML Design Basics

Let's consider the purpose of CNN and RNN designs.

The purpose of a convolutional network is to deal equally with regions in a multi-dimensional array of values in discrete samples of \R^n during an adaptive (learning) process.
The purpose of a recurrent network is to adapt to (learn) potentially complex temporal (time-wise) trends in potentially complex nonlinear systems.

Understand that the SSD class of algorithms do not do what natural visual systems do. They do not zoom in and out on independent objects within the network seamlessly. They cannot note that a base ball player is running to first base and a ball is coming from the catcher at the same time, requiring independent conceptual zoom operations within the neural network. This cannot be done with a zoom lens. That is why the Director of Photography is such a key role in movie making. The visual data must contribute well to the story telling, using lighting, camera orientation, panning, zooming, and depth of focus.

Although one can create several bicycle concept classes to cover various bicycle sizes, orientations relative to the optics, and distances away, there are limitations to this approach, which can be diminished with circuit parallelism in hardware. Multi-threading and serial evaluation can, depending on resources and patience factors, increase training time beyond what is practical. The challenge is to create seamlessness between low level concept classes of a bicycle in a frame as the bicycle angle and distance changes relative to the optical path.

Deeper into Details

The, "heavy distortion on the objects," could be a show-stopper if the root cause of the distortion mentioned is poor resolution in the time, horizontal, or vertical dimensions. The most significant and consistent image-oriented feature of a bicycle is two ellipses (not always circles) in close horizontal proximity and even closer vertical proximity — the two wheels. The wheels need to be recognizable.

Two general categories of networks were mentioned in the question, CNNs and RNNs, which, in general are the two most relevant overall categories of components in a visual system that recognizes motion. We have some nomenclature in the question, which begins the mathematical theory behind the design of the training of the networks and the real time requirements on those network components once the network components are trained.

  ... each ... frame ... has multiple detections of D1, D2, D3, ..., Dn

$$ D_i \land i \in {1, 2, ..., n} \\
\Downarrow \\
D_{xywh} $$

The above nomenclature presumably refers to a post-learning detection of concept classes $C_a, C_b, C_d$, where there is a many-to-one relationship between the above numeric indices for detections and these letter indices for the concepts of a bicycle to be recognized. Each concept class $C$ might correspond to a particular recognizable bicycle feature set given a particular range of distances to the optics and orientation of the wheels relative to the direction of light rays between the wheels and the camera. The designer, considering this correspondence cannot dismiss the turning of the front wheel. We cannot assume that the eccentricity of the visual representations of the two wheels will be the same, since the bicycle may be turning. Even in this more complex case, the ellipses are likely the two most differentiating features of bicycles in common scenes.

This may be a good time to point out that tricycle recognition may require the recognition of an entirely distinct set of concept classes.

Also notice that, if the optics (camera) is at a drastically different altitude than the wheels of the bicycles, such as images from a drone or a camera on a tall pole, the problem is a different one. This is the intensely effective quality of natural vision systems. Over millions of years, the kind of training that recognizes a bicycle from a drone video stream having only been trained to recognize a bicycle from ground level has emerged. Nature's ability to apply cognitive abilities to visual sequence recognition to use in trajectory prediction is not yet realized in software and hardware and the main problem in automated vehicle piloting and driving.

Two Different Output Requirements

There are two somewhat distinct problems that must be considered in analysis of the project requirements. The output could be either of these two, depending on whether counting bicycles is the primary goal or whether metrics of travel is its own independent objective.

Unit vector $\vec{r}$, presumably in $\mathbb{R}^2$ as a pixel vector, essentially a normalized vector of the first derivative of pixel position with respect to time. The center of the bike, in this case, would be based on the features of the bike in the field of view.
Unit vector $\vec{r}$, presumably in $\mathbb{R}^2$ as a geocoordinate unit vector, essentially a normalized vector of the first derivative of position with respect to time. The center of the bike, in this case, would be based on the features of the bike in geo-space without the altitude coordinate.

Approaches to AI Design

The common, but inefficient, artificial network approach is to locate the bicycle in each frame with a CNN and then use one of the progressive RNN types (either a GRU or a b-LSTM network) to recognize motion trends. One of the largest drawbacks is that you may have many concept classes that represent adjacent size-distance-orientation concepts (kernel based recognition models) of a bicycle to train into the CNN. If the bike is traveling toward or away from the optics at some angle, then the disappearance of the bike from $D_a$ and its appearance in $D_b$ needs to be construed as the contiguous motion of one bike. This is not an easy challenge but is heavily covered in the literature.

It is recommended to use web searches designed to search scholarly articles, not dummies guides, which are not reliable. There many academic publications that can be found with the search term, "Image recognition changing distance orientation." Looking at old articles from the 90s will provide a good historical context. Looking at new ones from the last three years will provide a survey of the current state of research.

Other Questions Within the Primary Question

The original types of recurrent networks are essentially for historical context. The dominant in-field recurrent network successes are often of the LSTM, b-LSTM, or GRU types.

The language (Scala, Python, Java, C, C++) is not particularly relevant if you are delegating the training to a GPU (which always runs C/C++ code),
so it may be unwise to consider reliability concerns as a primary driver for programming language selection.

Regarding, "How AI manages to count the objects," AI doesn't — not at the current state of technology. There is no one approach or algorithm across AI technology that dominates over all other approaches for all domains, into which bicycles can be plugged in and it works.

Currently, the AI engineer designs how the objects will be counted based on the characteristics of the objects to be counted, the meta-features of the incoming stream or data set, and the specifics of the recognition challenge. This is again because the wider capabilities of natural vision systems using the more sophisticated neural nets in animals and people has not yet been invented.

Final Recommendations Regarding System Design

The division between the use of kernels, in the CNN context, and the use of one of the recurrent network types is critical. If the engineer tries to delegate too much to the kernel, the above issue of bicycle distance to the optics and turning corners is exacerbated, because kernel operations do not lend themselves well to orientation and distance complexity. However, the CNN approach is excellent for the most upstream operations, such as edge detection and primitive object detection.

Let the recurrent network (of the more advanced types mentioned above) detect the bicycles as their distance and orientation in relation to the optical path changes, unless you have a sizable GPU farm that will perform CNN operations covering many distance and orientation ranges in parallel. Even if you do or have the patience of a saint, it may be best to delegate total bicycle recognition to the recurrent network anyway, since this is likely closer to the way natural systems do it and the modeling of bicycle travel between distance and orientation categories can be made more naturally seamless.

Recapping the comment above in this context, the problem complexity would be much lower if the bicycles were on a path, could not turn corners, and must travel either right to left or left to right within a speed range governed by the flow of traffic.

Response to Comments

Regarding Edit 1,

  Edit 1: The area of view under the camera is fixed. And we expect the bicycles to move from one entry side. Lets assume that the view and entry/exit is like shown in this video,

YouTube.com reports, "This video does not exist." That the camera is fixed, had been assumed in the writing of this answer because no camera trajectory described in the question. Had the expectation that bicycles will move from one entry side of the frame been included in the question, the answer would have addressed that case, but there was no hint of that requirement prior to Edit 1. Nonetheless, much of the content in this answer to the more general case still applies.

Regarding practicality, let's differentiate practical from prefabricated. The problem originally described before Edit 1 has no prefabricated solution into which one can plug bicycle data and tweak a few parameters to achieve success. In fact, the general interest on the web in seeking prefabricated solutions is usually met with the practical reality that such plug and play cases in machine learning are rare. Most of the time an approach that involves design and experimentation is usually the case.

Those that have hired and managed human beings know that even the hoped-for high level AI of the future, although possibly quite practical, may be as prefabricated as idealized. For instance, hiring an EE does not mean that the electrical engineering department will immediately see a practical improvement in throughput. Management, training, and workflow design will still be prerequisites to employee effectiveness. To have a practical and reliable bicycle counting, some comprehension of concepts to guide initial experimentation, design, and tuning of the training and use scenarios will likely be necessary.

If the author of the question has one of those very rare cases where bicycles travel in exactly one direction and in series, with no foot traffic, trikes, walking pedestrians, or pets, then AI is not necessary at all. A simple LED and photo-transistor with a passive low pass filter and a digital input to a counter circuit will perform a reasonable count. However, if two bicycles might pass in parallel, then we are back with the camera, the need for concept classes, and challenges much like those discussed in detail above.

Regarding steps to a solution, if approaching from the side of a technology to investigate, this answer includes a recommendable sequence, although there is usually quite a bit of overlap and occasional backtracking in actual practice. If this is not a learning exercises and the bicycle problem is one that is actually needed to operate in the field, then the above answer is the correct background to comprehend what it is the machine is required to perform. Following that comprehension would be the investigation of various designs and algorithms, beginning with a search of scholarly articles using the key phrases in the original answer.


What kind of AI technique can I use to play the "Lines" game? I am trying to find a good approach to create a computer player for the game "Lines" from gamious on Android. The concept of the game is pretty straightforward :

Lines is an abstract ‘zen’ game experience where form is just as important as function. Place or remove Dots to initiate a colourful race that fills a drawing. The colour that dominates the race wins.

The game starts with a drawing (that can be described as a set of "blank" lines, with connection to other lines). Dots of different colour are placed somewhat randomly on the lines. The player get a colour assigned. When the game start, paint start flowing from the dots and filling the (at first blank) lines of the drawing. You win if your colour dominates.
The game gives you different tools to win (the game starts when all of them have been used) :

[0 to 2] scissor to cut lines
[0 to 5] additional dot of your own color to place on the drawing
[0 to 4] enemy dots eraser
[0 to 3] additional straight lines to connect different part of the drawing

A quick example: the first image is the initial state of a round. "My" colour is the yellow (1 enemy = brown) and I have 4 tools (2 eraser and 2 lines). The second image shows the game running after I used the tools to put my colour in a winning position (yes, we can do better)

If I try to approach this as a classical optimization problem, things get messy pretty fast :

highly non-linear
high number of dimensions

AI seems to be the right way to go, but I would like your help to get in the right direction: what would be your approach to create an AI to play this game?
To limit the scope of this question, you can consider that I already have a data structure to represent the game initial state, the use of different tools and the game "physics". What I really want to do is finding how to create an AI which can learn how to efficiently use the tools.
Regarding my experience, I took 2 semesters of AI classes during the last year getting my engineering degree and have used non-linear optimization tools for a while: you can go technical, but I am not fully understand it.

Writing a game AI is difficult in general and it's even harder when there's no game to play. Like in your case.

While "Lines" is pretty neat, there's no (playing) opponent and there are no moves. apart from the setup. So what we have is an optimization problem where we need to get more point than the "opponents".

Game playing algorithms like Reinforcement learning or MCTS are IMHO out of questions as there's no game to play. Still, it's an interesting problem, though I can't see how to make any learning AI. Anyway, what you need first is a simulator and an optimizer choosing promising actions.

Some ideas concerning the actions

Some actions are trivial, e.g., the eraser as there are just a few dots to erase. Trying all combination should be good enough for the start.

The scissors can be in theory placed anywhere, but the only interesting places are just next to the oponent piece or a crossing. Additionally, you only need to consider crossings which gets "won" (reached first) by an opponent. Moreover, they only matter if they get won by an opponent having more points than you or blocking you.

The dots to insert should similarly by placed near an oponent or so that you just win a crossing.

The additional lines are worst, as they may be used to let you reach unconnected areas, or reach a crossing faster or simply give you additional points (then you want a long line somewhere).

A model

Most of the game can be seen as a non-directed graph with edges labeled by their length, where the crossing and the initial dots are nodes. The simulator doesn't need to work "pixel by pixel", but may instead jump to the next time, when a crossing gets reached or two "flows" meat each other. The points of a player grow linearly with time, where the slope is given by the number of their extending "flows".

So far, I could imagine modelling it as an Integer Linear Program, but with the additional lines, it's probably no more possible.

I'd probably let the optimization be simulator-guided, i.e., analyse whether I could get closer to winning by reaching a crossing sooner. The details are still unclear....


Should deep residual networks be viewed as an ensemble of networks? The question is about the architecture of Deep Residual Networks (ResNets). The model that won the 1-st places at "Large Scale Visual Recognition Challenge 2015" (ILSVRC2015) in all five main tracks:

  ImageNet Classification: “Ultra-deep” (quote Yann) 152-layer nets
  ImageNet Detection: 16% better than 2nd
  ImageNet Localization: 27% better than 2nd
  COCO Detection: 11% better than 2nd
  COCO Segmentation: 12% better than 2nd
  Source: MSRA @ ILSVRC & COCO 2015 competitions (presentation, 2-nd slide)

This work is described in the following article:

  Deep Residual Learning for Image Recognition (2015, PDF)

Microsoft Research team (developers of ResNets: Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun) in their article:

  "Identity Mappings in Deep Residual Networks (2016)"

state that depth plays a key role:

  "We obtain these results via a simple but essential concept — going deeper. These results demonstrate the potential of pushing the limits of depth."

It is emphasized in their presentation also (deeper - better):

  - "A deeper model should not have higher training error."
  - "Deeper ResNets have lower training error, and also lower test error."
  - "Deeper ResNets have lower error."
  - "All benefit more from deeper features – cumulative gains!"
  - "Deeper is still better."

Here is the sctructure of 34-layer residual (for reference):

But recently I have found one theory that introduces a novel interpretation of residual networks showing they are exponential ensembles:

  Residual Networks are Exponential Ensembles of Relatively Shallow Networks (2016)

Deep Resnets are described as many shallow networks whose outputs are pooled at various depths.
There is a picture in the article. I attach it with explanation:

   Residual Networks are
  conventionally shown as (a), which is a natural representation of
  Equation (1). When we expand this formulation to Equation (6), we
  obtain an unraveled view of a 3-block residual network (b). From this
  view, it is apparent that residual networks have O(2^n) implicit paths
  connecting input and output and that adding a block doubles the number
  of paths.

In conclusion of the article it is stated:

  It is not depth, but the ensemble that makes residual networks strong.
  Residual networks push the limits of network multiplicity, not network
  depth. Our proposed unraveled view and the lesion study show that
  residual networks are an implicit ensemble of exponentially many
  networks. If most of the paths that contribute gradient are very short
  compared to the overall depth of the network, increased depth
  alone can’t be the key characteristic of residual networks. We now
  believe that multiplicity, the network’s expressability in the
  terms of the number of paths, plays a key role.

But it is only a recent theory that can be confirmed or refuted. It happens sometimes that some theories are refuted and articles are withdrawn.

Should we think of deep ResNets as an ensemble after all? Ensemble or depth makes residual networks so strong? Is it possible that even the developers themselves do not quite perceive what their own model represents and what is the key concept in it?

Imagine a genie grants you three wishes. Because you are an ambitious deep learning researcher your first wish is a perfect solution for a 1000-layer NN for Image Net, which promptly appears on your laptop.

Now a genie induced solution doesn't give you any intuition how it might be interpreted as an ensemble, but do you really believe that you need 1000 layers of abstraction to distinguish a cat from a dog? As the authors of the "ensemble paper" mention themselves, this is definitely not true for biological systems.

Of course you could waste your second wish on a decomposition of the solution into an ensemble of networks, and I'm pretty sure the genie would be able to oblige. The reason being that part of the power of a deep network will always come from the ensemble effect.

So it is not surprising that two very successful tricks to train deep networks, dropout and residual networks, have an immediate interpretation as implicit ensemble. Therefore "it's not depth, but the ensemble" strikes me as a false dichotomy. You would really only say that if you honestly believed that you need hundreds or thousands of levels of abstraction to classify images with human accuracy.

I suggest you use the last wish for something else, maybe a pinacolada.


How can we design the mutation and crossover operations when the order of the genes in the chromosomes matters? Consider an optimization problem that involves a set of tasks $T = \{1,2,3,4,5\}$, where the goal is to find a certain order of these tasks.
I would like to solve this problem with a genetic algorithm, where each chromosome $C = [i, j, k, l, m]$ corresponds to a specific order of these five tasks, so each gene in $C$ corresponds to a task in $T$.
So, for example, $C = [1,3,5,4,2]$ and $C' = [1,5,4,2,3]$ would be two chromosomes that correspond to two different orders of the tasks.
In this case, how could we design the mutation and cross-over operations so that these constraints are maintained during evolution?
The genetic algorithm should produce the three best chromosomes or order of tasks.

If I understood correctly, your problem is about finding the optimal way to execute a series of tasks in order to maximize the results, using Genetic Algorithms.

In few words, you're trying to solve the salesman problem.

If I am correct, you're looking for Crossover and Mutation algorithms that allow you to work with ordered sets of elements. For these scenarios you usually go for the classic PMX
(Partially Mapped Crossover) and Interchange Mutation. But, there are plenty of other crossover algorithms you can use OX1, OX2 (both variants of the Order Based Crossover), Shuffle Crossover, Ring Crossover, etc. Let's start from the mutation, that is easier.

For simplicity I'll represent the ordered genome like an array of integers: int[] genome = {1, 2, 3, 4, 5};

Interchange mutation

The concept is pretty basic: to mutate an ordered genome you just swap two elements. Easy.

    public int[] InterchangeMutation(int[] genome)
    {
        int i1 = random.Next(0, genome.Length);
        int i2 = random.Next(0, genome.Length);

        var copy = genome.ToArray(); //just making a copy here
        copy[i1] = genome[i2];
        copy[i2] = genome[i1];

        return copy;
    }

PMX Variation Crossover

This is a bit more complicated as we have to take repetitions into account. From experience, I like to use this variation of the Partially Mapped Crossover. It is way easier to implement than the original one (you can find the paper online) but it will cost some more computational complexity. Longer the genome, higher the price you will pay.

Start by selecting two parents to use for the crossover.
From the first parent (P1) select a random section that will be passed over.
For the remaining values:
3A. If they are not in the copied section, take them from P2
3B. If they are in the copied section, take them from P1
3C. End up filling the gaps with the missing values in the order they are in P1

 public int[] PMX2Crossover(int[] P1, int[] P2)
    {
        //Initializing child genome
        int[] child = new int[P1.Length];
        for (int i = 0; i < P2.Length; i++) child[i] = -1;

        //Step1: getting random section to copy over
        int i1 = random.Next(0, P1.Length);
        int i2 = random.Next(0, P1.Length);

        //Step 2: Copying over section from P1
        for (int i = Math.Min(i1, i2); i < Math.Max(i1, i2); i++) child[i] = P1[i];

        //Step 3A: Copying values from P2
        for (int i = 0; i < P2.Length; i++) if (child[i] ==-1 && !child.Contains(P2[i])) child[i] = P2[i];

        //Step 3B: Copying values from P1
        for (int i = 0; i < P2.Length; i++) if (child[i] == -1 && !child.Contains(P1[i])) child[i] = P1[i];

        //Step 3C: Copying remaining values from P1
        int emptyGene = child.IndexOfFirst(-1);
        while (emptyGene != -1)
        {
            child[emptyGene] = FirstMissingGene(P1, child);
            emptyGene = child.IndexOfFirst(-1);
        }

        return child;
    }

    private int FirstMissingGene(int[] parent, int[] child)
    {
        foreach (var gene in parent) if (!child.Contains(gene)) return gene;
        return -1; // should never get here
    }

You can lower down the complexity of the crossover to O(n) (from O(n*n)) simply using a hashmap that keeps track of the genes already added to child.

To get the first child call PMX2Crossover(P1, P2); and for the second just swap the parents PMX2Crossover(P2, P1);

Hope this helps you.

Source: I have been a bachelor professor of AI for a period.


Further Normalization of Standardized data - ANN I want to develop a regression model using the artificial neural network. For developing such a model I use standardised ( z-score normalised ) data.
given below is the sample data set. Here MAX is the real data But I am using MAX-ZS (these values are continues)

So my question is while developing the model do I have to perform further normalization such as Min-Max scaling on my training data?
Any Kind of help is appreciated!

Data scaling or normalization is a process of making model data in a standard format so that the training is improved, accurate, and faster.

So you just have to scale the data once. Doesn't matter what scaler you are using. Just make sure to initialize the scaler with the training data and then use the same parameters to scale the test data.

The z-score normalized data (MAX-ZS) can be used directly to train the network.


Can we derive the distribution of a random variable based on a dependent random variable's distribution? In the diagram below, there are three variables: X3 is a function of (depends on) X1 and X2, X2 also depends on X1. More specifically, X3 = f(X1, x2) and X2 = g(X1). Therefore, X3 = f(X1, g(X1)).

If the probabilistic distribution of X1 is known, is it possible to derive the the probabilistic distribution of X3?

Yes you can, provided you know about $f$ and $g$. Expression $X3 = f(X1, g(X1))$can be written as $X3 = h(X1)$ where $h$ takes into account both $f$ and $g$. After this finding the PDF is simple by differentiating the CDF:
$$ F_{X3} (x3) = P(X3 \leq x3) =  P(h(X1) \leq x3) = P(X1 \leq h^{-1}(x3))$$

$$ \frac {d F_{X3} (x3)}{dx3}  = \frac {d P(X1 \leq h^{-1}(x3))}{dx3} = f_{X3}(x3)$$

NOTE: The conventions followed are the same as used in the field of Probablity

(Take care of the function inversion step in non-monotonic cases)

Check these lectures.


What do we mean by 'principal angle between subspaces'? I came across the term 'principal angle between subspaces' as a tool for comparing objects in images. All material that I found on the internet seems to deal with this idea in a highly mathematical way and I couldn't understand the real physical meaning behind the term.
I have some knowledge of linear algebra. Any help to understand the physical significance of this term and its application in object recognition would be appreciated.

Let's consider the case where you have two photos, one base photo and one other photo which is a scaled version of the base photo. Consider then that you could create a 'mapping' from the base photo to the scaled photo as defined by a set of vector changes for each pixel in the base photo.
That is to say, if you had a pixel at point (0,0) in the base photo it would be in position (0,0) on the scaled photo. If you had a pixel at point (0,1) on the base photo, it would be at point (0,s) on the scaled photo, where s is the scale factor between them. The same would be true for (1,0) mapping to (s,0) and (1,1) mapping to (s,s).
We can understand each of these pixels as a subspace on the vector space of "all possible pixels" (possibly with an included vector for the 'value' of each pixel as a third+ dimension, based on how it's represented). Note though, that they are different subspaces. There are vectors that exist in the scaled picture for scale factors greater than 1 which are not in the original picture. For scale values less than one, there are vectors which are in the original picture which are not in the scaled picture. For scale vector equal to one, it's basically the identity and they're the same picture.
What's more, we can do the same for rotation. If you have base photo and rotated photo, then there is a mapping from one pixel on the base photo to one pixel on the rotated photo, I'll leave the math up to you. The same is further true for translating, skewing, and even changing the colors of the image. Each one denotes a type of mapping that you could make from one subspace to another.
So what does this mean? Well, one thing that you could do is take two images, one "base" photo and two "scaled" photos and determine "how different are these two photos from the base photo"? That is, if the first scaled photo has a scaling factor of 10 and the second has a scaling factor of 100, then you could say that the larger scaled-photo is "further away" from the base photo. That is, we can compare the two mappings together to determine which mapping is closer to the original.
What's more, these mapping functions are often composable and commutative. That is if you do a "scale-then-rotate", that is very similar if not exactly identical to doing "rotate-then-scale". Let's take a look at why this is. Let's consider that we take a unit vector from either one. We then apply "scale" which scales both the x and y components, and then rotate this unit vector about some axis (let's assume the origin, though maybe there's a translation that happens as well). You will then yield a new vector which represents all of these applied changes. What's more, we can do this to both of the "scale-then-rotate" function and the "rotate-then-scale" function.
Now we have two vectors, and we want to determine if they are the same. Well, one way to do that is to calculate the angle between two vectors. If the angle is 0, then we can say that the vectors are the same (with the assumption that they are also of the same magnitude). In this case, both would yield the same vector, the angle between them is 0, so we can say they are the same.
So then how do we get back to the question of "how different are these two objects"? Well, one way that we could do it is to attempt to answer the question "What set of composed functions maps from one object onto the other one?". If we can create a new "vector" which is the representation of the full set of changes from one picture to the other, we can then attempt to determine how different these two objects are, as represented in the dimensional spaces of their image.
But the question that you may then ask is "why is this useful"? Well, imagine that you are a neural network. You've been given a picture and tasked with answering the question "Is this a cat?". How would you go about answering this? Well, one thing as a neural net that you've learned is "cats have ears". You have a canonical representation of a "cat ear" encoded in the weights of your neural network, so you go about looking at different subsets of the picture to say "is this a cat ear"? You can do this, for example, by taking a cropped section of the picture, determining the angle and dimensions apart that cropped section is from your canonical "cat ear" is, and if the angle between your canonical "cat ear" and your sample is below some amount, you can say with confidence that what you have found is a "cat ear". A couple cat ears, eyes, a nose, some fur, and a tail (though maybe the tail is occluded), and you can be confident that you have found a "cat".
(Note: This is an oversimplification and this is usually done in deep networks by breaking pictures out into a continually composing series of features, for instance first "lines" vs "curves" vs "textures", which then yield "circles" vs "rectangles" vs "cross-hashes", which then yield "eyes" vs "fur" vs "skin", etc. Each different layer could be implemented with its own comparison in this same method though!)


Why are policy iteration and value iteration studied as separate algorithms? In Sutton and Barto's book about reinforcement learning, policy iteration and value iterations are presented as separate/different algorithms.
This is very confusing because policy iteration includes an update/change of value and value iteration includes a change in policy. They are the same thing, as also shown in the Generalized Policy Iteration method.
Why then, in many papers as well, they (i.e. policy and value iterations) are considered two separate update methods to reach an optimal policy?

Policy iteration is made up of two steps. The first is a full policy evaluation, where a value function is calculated for the current policy. The second is policy improvement, where the policy is made greedy with respect to the value function.
Value iteration looks to speed things up by stopping policy evaluation after one iteration, make the policy greedy with respect to that value function, and repeat until convergence.
Clearly, these are two different algorithms, hence why they are considered to be different. They are, however, very closely linked, which is why you might consider them to be 'the same thing'. I guess you could say they belong to the same family of algorithm.


Are there any machine learning techniques to detect coding standard violations? Are there any machine learning techniques (such as deep learning or evolutionary algorithms) to detect coding standard violations? Which one would be more suitable?
I don't have any specific programming language in mind, but something similar to PHP_CodeSniffer (following these standards), but instead of using hardcoded rules, the algorithm should learn good techniques, but I'm not sure based on what training data. How would you approach the training session, any suggestions?

If the system claims that a piece of code has violated standards, then to be useful to the programmer, it really needs to provide more information than just a 'yes/no' classifier: you need some form of explanation about why it is claimed to be wrong.

Clearly ANNs aren't much use for that.

If I were tackling such a problem (and my suspicion is that a lot of effort could be spent trying and failing to reproduce coding standards which are already well-understood), then my inclination would be to use a more explicitly rule-based representation.

Possibilities include:

Genetic Programming
Learning Classifier Systems
Decision Trees

The ever-useful "Essentials of Metaheuristics" has a whole section on the evolution of rulesets. Obviously, nothing prevents you from initializing the evolutionary process with rules known to be useful.

As I point out here, with our current AI algorithms, the success of an  approach is very sensitive to human expertise/effort in feature selection/preprocessing, choice of training set etc, so creative experiment with this is vital.

Training set: how about two sets of negative and positive examples, consisting of (features extracted from) bad code and from a refactored version (respectively)?

One elementary choice of features would be to apply a bunch of code complexity metrics and have the learning algorithm combine those. The plus side of working in such a numeric domain is that the learning algorithm might readily find a gradient to exploit. The downside is that the rules (which are then likely of the form if mcabe > 2.8 etc) are still not as informative as might be desired.

For more complex rules (e.g. requiring if elseif else) you may want to extract your features from the abstract syntax tree. You could in principle use the entire tree but to my knowledge ML on graph and tree structures is still in relative infancy.


The mathematics in the CBOW and Skip-Gram models this is my first question on AI Stack Exchange. I am a mathematics student who is learning NLP so I have paid a high amount of attention on the mathematics used in the subject, but my interpretations may or may not be right sometimes. Please correct me if any of them are incorrect or do not make sense.
So I have learned CBOW and Skip-Gram models. I think I have understood the CBOW model, and here is my interpretation: First, we fix a number of neighbors of the unknown center word which we would like to predict; let the number be $m$. We then input the original characteristic vectors (vectors of zeros and ones only) of those $2m$ context words. By multiplying those vectors by a matrix, we obtain $2m$ new vectors. Next, we take the average of those $2m$ vectors and this is our hidden layer, namely $v$. We finally multiply $v$ with another matrix, and that is the "empirical" result.
I tried to follow the logic to Skip-Gram similarly, but I have been stuck. I understand that Skip-Gram is kind of a "reversal" of CBOW, but the specific steps have given me a hard time. So in Skip-Gram, we only have a center word, and based upon that we are trying to predict $2m$ context words. By similar steps, we obtain a hidden layer, which is again a vector. The final process also involves multiplication with a matrix, but I don't know how we can get $2m$ new vectors based upon one, unless we have $2m$ different matrices?
I appreciate any insights or answers for my question. Thanks!

The following figure from this article can be helpful:

This figure represents "Skip-Gram model structure. Current center word is 'passes'".


How to automatically detect and correct false information in columnar data? I'm working on data cleaning and I'm stuck. I have a data set with 3 columns: id, age, and weight.

Supposing I have an entry:

id:1 | age:3 (years)  | weight: 150 (kg)

How can we detect that the information is wrong, assuming I have a thousand lines?

And how can I correct it (using Python)?

Is there any function in Python that I can use or should I use machine learning techniques?

While I can see that there are some heuristics that can tell you whether an entry is 'weird', I don't see any way that you can correct this. Where would you get a correct value from?

I would perhaps start with a statistical analysis, looking at the distribution of values to get an idea of the state the data is in. From this you can then already see some values that wouldn't make sense (this depends very much on what data it is: census data will include toddlers, but credit card applications wouldn't).

Your best bet is to encode some constraints, such as a minimum/maximum range for each value, and perhaps some correspondences, such as a maximum weight for a certain age range. This you can do with simple conditional statements in Python.

You can then flag up those entries which don't look quite right. What you do with them depends on your application. You can filter them out, or manually change the values to something that is consistent with possible values.

Machine Learning techniques would not be able to do anything better than that.


How would an AI work out this question? I am trying to create an AI that makes reasonable guesses at truths of statements. However...

Human: "Prove that no number exists which is one more than a billion."

AI: "Is it true for the number 1? No. I am 1% sure the statement is correct."

AI: "Is it true for the number 2? No. I am 2% sure the statement is correct."

...

AI: "Is it true for the number 999,999? No. I am 99.99% sure the statement is correct."

AI: "Having tested a large number of examples. I conclude that the statement is correct."

Human: "The statement is wrong: one billion and one."

What do you think has gone wrong?

I don't think that the "try all the numbers" approach is very representative, because I'm not sure whether or not the agent that uses that approach can be considered by any means AI.

There is no "intelligence" in just checking numbers to try to prove the statement. An agent that is considered to be intelligent should apply a more intelligent approach.

This becomes more evident because the question aims at exploiting the lack of scalability of the agent's strategy. If the question was "Prove that no number exists which is one more than 5", then the agent would have no trouble in finding the correct answer.


How are IOUs for ground truth boxes in YOLO calculated? I know how IOU works during detection. However, while preparing targets from ground-truth for training, how is the IOU between a given object and all anchor boxes calculated?
Is the ground truth bounding box aligned with an anchor box such that they share the same center? (width/2, height/2)
I think this is the case but I want to hear from someone who has better knowledge of how training data is prepared for training in YOLO.

My assumption was correct: the ground truth bounding box is aligned with an anchor box such that they share the same center

In other words, only the widths and heights are used to calculate the ground truth IOU.


Divide classes into truncated and non-truncated objects At the moment I am working on a vehicle counting & classification project.
For a specific part in the project I need to get back only the completely visible vehicles from my input data (images).  I am wondering if this could be done (more) automatically in the following way:

zoom in such that only approximately one van would be visible
divide the vehicles into two categories: truncated and non-truncated
train on these two classes
After training and testing, use the model to find the completely visible vehicles.

So the main question is, is it possible that this would give sufficient results or should I try to find another solution?

It could work. I think itll be hard to find labelled bounding box data describing full/truncated cars so this is a good idea for self labelling.

Theres probably also some other ways for you to get labelled data without the crowd sourcing or self labelling. Take a large labelled car dataset with bounding boxes (COCO has some, and if your good with aerial imagery theres alot more out there), and just enforce that any car bounding boxes that are touching the boundary are truncated, while ones fully encompassed in the imaged are not, since with high probability that will be the case.

Good luck!


What is Federated Learning? How would you explain Federated Learning in simple layman terms for a non-STEM person?
What are the main ideas behind Federated Learning?

The analogy is to a federal system of government. In a federation, smaller pieces follow the direction of a higher piece. In federated machine learning, you give your data for processing to the higher machine. The federation in this analogy is a collection of smaller computers. The central computer breaks up your data and gives portions of it to each smaller computer. When those computers are done they return the results and the central computer reassembles them into a single model.
The main idea is distributed processing. Some benefits are:

Cost: it may be cheaper to operate multiple inexpensive computers rather than fewer more expensive computers.
Privacy: if this is sensitive data like healthcare records then perhaps you don't want all of one person's data in a single place where the wrong person can grab it.


CNN output generally has more than one category in one-hot categorization? I'm a bit of a CNN newbie, and I'm trying to train one to image classify pictures of pretty similar looking particles. I'm making the inputs and labels by hand from a set of 48x48 grayscale images, and labeling them with a one-hot vector based on their position in the sequence (for example, the 400/1000th image might have a one-hot in the 4th position if I have 10 categories in the run). I'm using sigmoidal output activation and categorical cross entropy loss. I've played around with a few different optimizers, as well. I'm implementing in python keras.

Unfortunately, although I have pretty good accuracy numbers for the training and validation, when I actually look at the outputs being produced, it generally gives multiple categories, which is not at all what I want. For example, if I have 6 categories and a label of 3, it might give the following probability vector:

[ .99 .98  1.0  .99  0.02  0.05 ]

It was my understanding that categorical cross entropy would not allow this type of categorization, and yet it is prevalent in my code. I am under the impression that I'm doing something fundamentally wrong, but I cant figure out what. Any help would be appreciated.

When you use sigmoid activation it is applied independently to all outputs so outputs won't sum up to 1. Sigmoid is usually used for binary logistic regression where you only have 2 classes. In your case you should use softmax activation at the output which will squash outputs to range [0,1] and additionally make them sum up to 1.


How can I understand this statement about RNNs and hidden layers? In the lecture, there was a statement:

  Recurrent neural networks with multiple hidden layers are just a special case that has some of the hidden to hidden connections missing.

I understand recurrent means that can have connections to the previous layer and the same layer as well.

Is there a visualization available to easily understand the above statement?

I assume the statement was made for Elman recurrent neural networks, because as far as I know, that is the only type of neural networks for which that statement is valid.

Let's say we have an Elman recurrent neural network with one input neuron, one output neuron and one hidden layer with two neurons.

In total there are 10 connections. As the image shows, neuron A receives the combined previous output of both neuron A and B as input. The same goes for neuron B.

This is not the case when we split the neurons up into multiple layers; the context neuron(s) are only used by neurons that are in the same layer. Let say we now use multiple hidden layers and keep the amount of neurons the same. In total there are 7 connections now (image below). That is 3 less than in the first example, which has only one hidden layer. So which connections do we miss? That is shown in the bottom image. (I had to paste these two images together in one image because my reputation only allows me to post 2 links)

Please note the cross; the connection between neuron A and B is not there in the first image, because it would be some kind of random recurrent connection.

The first and the last image are exactly the same. I think that if you compare the first and the last image that you agree that the statement is true.


How to calculate the output of this neural network? What is the output value of the network for these inputs respectively, and why?
(Linear activation function is fine.)

[2, 3][-1, 2][1, 0][3, 4]

My main question is how you take the 'backwards' directed paths into account.

The neural Network in the image is a "Recurrent Neural Network"(RNN). Because of the connection leading backward from h10 to h01, h10 has to be a "memory node" (mn), meaning it can store its value from the previous input. The basic functionality of an RNN can be seen in this animation:

Your example:
In the beginning, the storage of the mn is initialized with a value, probably 0.
Now the first input is fed into network:

i0 = 2
i1 = 3
h00 = (i0 * 0.4) = 0.8
h01 = (i1 * -0.9) + ("the stored value of h10" * 1.2) = -2.7
("the stored value of h10" in the first run is 0.)
h10 = (h00 * 0.85) + (h01 * -0.2) = 1.22
out = (h10 * 0.3) + (h01 * 0.1) =  0.096

Now you can feed the next input through the network and use -2.7 as "the stored value of h10" and so on. You can also add an activation function as you would for any other NN.


What is the best machine learning algorithm to select best 3 variable combinations? I have 10 variables as like below

$V_1=1$, $V_2=2$, $V_3=3$, $V_4=4$, $V_5=5$, $V_6=6$, $V_7=7$, $V_8=8$, $V_9=9$ and $V_{10}=10$

Note: Each variable can have any value
Now I want to select the best 3 variables combination as like below

$V_1V_3V_4$ or $V_{10}V_1V_7$ or $V_5V_3V_9$ etc.

The best combination is nothing but the sum of the values of 3 variables in the combination.
Example:

Combination 1($V_1V_2V_3$) : 1+2+3 $\Rightarrow$ 6
Combination 2($V_8V_9V_{10}$) : 8+9+10 $\Rightarrow$ 27

In the above example Combination 2($V_8V_9V_{10}$) has the highest sum value. So the Combination 2($V_8V_9V_{10}$) is the best combination here.
Like this, if I have a large number of variables means which machine learning algorithm selects the best combination in all the sense.
Suggest me the best machine learning algorithm for selecting the best variable combinations. Thanks in advance.

What you have is an instance of a local search or black-box optimization problem.
There are many AI algorithms to do this, though as commenters have noted, with only 10 binary variables, you're happier to just look at all $2^{10}$ of them.
If you had, say, 60, then looking at them all is no longer a good idea ($2^{60}$ is very large).
The simplest BBO technique is called hill climbing. It looks like this:

Pick a random subset of the variables to be true.
For each variable, determine the effect of flipping that variable from true to false in the current assignment.
Select the 'flip' that leads to the largest value of your function.
If the 'flip' has a larger value of your function than the current assignment, GOTO 2.
Otherwise, return the current assignment.

This amounts to: 'Pick an assignment at random, then keep making small changes that improve it until you can't anymore. Claim that this is the best solution.'
Clearly, this algorithm will not recover the globally best solution with certainty, but you could run it many times, and eventually, it will find it with high probability.
A complete algorithm, that will always uncover the global optimum, is simulated annealing. This algorithm has a temperature parameter, usually denoted T, which changes according to a temperature schedule. If you start T large 'enough', and decrease it slowly 'enough', then it will find the global optimum. Unfortunately, 'enough' depends a lot on the shape of the function you are optimizing, which you generally cannot observe. There are some common guesses for T, like multiplying it by, e.g. 0.9999 at each step in the search, that seem to work okay in practice.
Both of these algorithms are widely available in packages.


What's the threshold to call something 'machine learning'? For example, if I use some iterative solvers to find a solution to a non-linear least squares problem, is that already considered machine learning?

T. Mitchell defines machine learning in "Machine Learning" book as

a computer program is said to learn from experience  concerning some class of tasks  and performance measure , if its performance at tasks in , as measured by , improves with experience

Hence, based on the above definition, we can't say a machine learning method to every iterative method.
In your specific example, it is just a non-linear solver such as the Newton method to finding roots.
However, you should notice that a non-specific machine learning method can be used in the learning process. For example, you might need some numerical methods to compute the measure $P$ (in the above definition). But, we can't say that the specified method is a machine learning method.


Extracting Descriptors and feature points for 3d mesh I'm programming my work with python, and I have a mesh and I want to extract 3d descriptors and feature points from it( trying to work on multi-scale strategy) , to visualize them later on the mesh,

What I'm asking about, is references, guidelines, anything which could benefit me with this situation

The main work I'm trying to do, is to reach matching stage, where I could find one-to-one correspondence.

The work was done through the following:

Extract feature points using a Detector
Extract the descriptor for these feature points
Make matching using similarity measure between two descriptors from two meshes.


How create an AI that continuously adapts to different users? I want to make an AI with deep learning which can adapt itself from user to user.

Let's say we have food combiner AI which suggests a food to eat with another food as you give as input. This is the most personalized case I found to ask here. For example the AI suggested a food for me. However, the food AI suggested for me might not be good choice for another person. So another person will let the AI know like "I don't like that food to eat with this. Etc. When the user let the AI know that, It should affect AI's further combination food suggestions.

How can I build that AI? Where should I start from? which area or topics should I research?

Assuming you have enough training data and representational capacity, you can give each user a unique identifier and concatenate that with the other inputs to the neural net. As users give more feedback, the network will learn in the usual way since each situation (e.g. food-user pair) is represented by a unique input.

I'd consider this a general, brute-force approach, and it may not be suited to your application. Others' links to recommendation systems might be more useful. It depends on what your task is and what your constraints are.


How to estimate the capacity of a neural network? Is it possible to estimate the capacity of a neural network model? If so, what are the techniques involved?

VC dimension
A rigorous measure of the capacity of a neural network is the VC dimension, which is intuitively a number or bound that quantifies the difficulty of learning from data.
The sample complexity, which is the number of training instances that the model (or learner) must be exposed to in order to be reasonably certain of the accurateness of the predictions made given some data, is proportional to this number.
The paper VC Dimension of Neural Networks (1998) by Eduardo D. Sontag provides a good introduction to the VC dimension of neural networks (even though these concepts are quite abstract and you may need to read them several times to fully grasp them). The information in this answer is highly based on that paper.
Shattering and VC dimension
In section 2, Concepts and VC Dimension, he describes the basic concepts behind the VC dimension (not only for neural networks), such as the concept of shattering (i.e. what does it mean for a set of sets to shatter another set?), which is a well-known concept in computational learning theory and is used to define the VC dimension (see definition 2), so you definitely need to get familiar with this concept to understand the VC dimension and, therefore, the capacity of a neural network (calculated with the VC dimension).
VC dimension of functions and neural networks
He then provides an equivalent definition of the VC dimension but for functions (equation 6). Given that neural networks represent functions, then we can also define the VC dimension of a neural network. A specific combination of weights of neural networks represents a specific function, for which the VC dimension can be defined. To be more precise, a parametrized function (and a neural network) can be denoted as
$$
\beta : \mathbb{W} \times \mathbb{U} \rightarrow \mathbb{R}
$$
where $\mathbb{W} = \mathbb{R}^p$ and $p$ is the number of weights (or parameters) of the neural network, $\mathbb{U}$ is the input space and $\mathbb{R}$ the output space. So, in this case, $\beta$ can also represent a neural network, with a certain parameter space $\mathbb{W}$, an input space $\mathbb{U}$ and an output space $\mathbb{R}$.
The vector $\mathbf{w} = (w_1, \dots, w_p) \in \mathbb{W}$ represents a specific combination of weights of the neural network, so it represents a specific function. The set of all functions for each choice of this weight vector can be denoted as
$$
\mathcal{F}_{\beta} = \{ \beta(\mathbf{w}, \cdot) \mid \mathbf{w} \in \mathbb{W} \}
$$
The VC dimension (VCD) of $\beta$ can then be defined as
$$
\text{VCD}(\beta) := \text{VCD}(\mathcal{F}_{\beta})
$$
Therefore, the VC dimension is a measure of the capacity of a neural network with a certain architecture. Moreover, the VC dimension is equivalently defined for a certain set of functions associated with a neural network.
How to calculate the VC dimension?
To calculate the actual VC dimension of a neural network, it takes a little bit of more creativity. Therefore, I will just report the VC dimension of some neural networks. For more details, you should fully read the cited paper (more than once) and other papers and books too (especially, the ones described in this answer, which provide an introduction to CLT concepts).
VC dimension of a perceptron
The VC dimension of a perceptron is $m + 1$, where $m$ is the number of inputs. Given that a perceptron represents a linear and affine function, the VC dimension of the perceptron is also equal to the number of parameters. However, note that, even though the VC dimension of the perceptron is linear in the number of parameters and inputs, it doesn't mean the perceptron can learn any function. In fact, perceptrons can only represent linear functions. See section 3.1 of VC Dimension of Neural Networks for more details.
VC dimension of a single hidden layer neural network
Let $n$ be the number of hidden units, then the VC dimension of a single hidden layer neural network is less than or equal to $n+1$. See section 3.2 of VC Dimension of Neural Networks for more details.
VC dimension of multi-layer neural networks with binary activations
The VC dimension of multi-layer neural networks (MLPs) with binary activations and $p$ weights (or parameters) is $\mathcal{O}(p \log p)$. See theorem 4 (and related sections) of the paper VC Dimension of Neural Networks for more details.
VC dimension of MLPs with real-valued activations
The VC dimension of MLPs with real-valued activations is no longer bounded by $\mathcal{O}(p \log p)$ and can be exponential in the number of parameters. See section 5.3 of VC Dimension of Neural Networks.
VC dimension of MLPs with linear activations
The VC dimension of MLPs with linear activations is $\mathcal{O}(p^2)$.  See theorem 5 of the paper VC Dimension of Neural Networks.
Notes
The VC dimension is often expressed as a bound (e.g. with big-O notation), which may not be strict.
In any case, the VC dimension is useful because it provides some guarantees. For example, if you use the VC dimension to describe an upper bound on the number of samples required to learn a certain task, then you have a precise mathematical formula that guarantees that you will not need more samples than those expressed by the bound in order to achieve a small generalization error, but, in practice, you may need fewer samples than those expressed by the bound (because these bounds may not be strict or the VC dimension may also not be strict).
Further reading
There is a more recent paper (published in 2017 in MLR) that proves new and tighter upper and lower bounds on the VC dimension of deep neural networks with the ReLU activation function: Nearly-tight VC-dimension bounds for piecewise linear neural networks. So, you probably should read this paper first.
The paper On Characterizing the Capacity of Neural Networks using Algebraic Topology may also be useful and interesting. See also section 6, Algebraic Techniques, of the paper I have been citing: VC Dimension of Neural Networks.
The capacity of a neural network is clearly related to the number of functions it can represent, so it is strictly related to the universal approximation theorems for neural networks. See Where can I find the proof of the universal approximation theorem?.


Why do we regularize the variational autoencoder with a normal distribution? When we define the loss function of a variational autoencoder (VAE), we add the Kullback-Leibler divergence between the sample taken according to a normal distribution of parameters:

$$ N(\mu,\sigma) $$

and we compare it with a normal distribution of parameters

$$ N(0,1) $$

My intuition is that it is clever having samples taken from a distribution centered around zero, but I don't understand why we want that examples are taken with a normal distribution.

If you are mathematically inclined, here is an article that discusses the reasoning.

What I get as a take away is that the VAE forces the learned latent space to be Gaussian due to the KL divergence term in the loss function. So, now we have a known distribution to sample from to create input vectors to feed to the decoder, to produce say images of dogs, if the VAE was trained on images of dogs. As you sample from the distribution, you will produce images of different types of dog images.

I assume a different type of distribution could be selected if one uses the proper loss function for that type of distribution, that is, the loss function which would measure the difference in the distribution of the latent space and the desired distribution.

KL divergence is the loss function that forces the latent space distribution to be Gaussian. If you do not "restrict" the latent space, as is the case with a regular autoencoder, you have no idea what kind of vector to select as an input to the decoder to produce a dog image. Without restriction, there are $2^n$ (where $n$ is the dimensions of the latent space) possible vectors you could choose from. Chances of selecting one that produces a dog image would be minuscule.

Well, I hope this helps. I no longer am mathematically proficient (age 75), so I hope my interpretation of the math is correct.

A VAE tends to produce blurry images because there are two terms in the loss function. One term is trying to make the output look like the input while the KL loss term is trying to restrict the latent space distribution. GANs ( generative adversarial networks) don't have this conflict, so they produce much high-quality images.


When is deep learning overkill? For example, for classifying emails as spam, is it worthwhile - from a  time/accuracy perspective - to apply deep learning (if possible) instead of another machine learning algorithm? Will deep learning make other machine learning algorithms like naive Bayes unnecessary?

It's all about Return On Investment. If DL is "worth doing", it's not overkill.

If the cost of using DL (computer cycles, storage, training time) is acceptable, and the data available to train it is plentiful, and if the marginal advantage over alternative algorithms is valuable, then DL is a win.

But, as you suggest, if your problem is amenable to alternate methods, especially if it offers a signal that matches up well with classic methods like regression or naive Bayes, or your problem requires explanation of why the decision boundary is where it is (e.g. decision trees), or if your data lacks the continuous gradients needed by DL (especially, CNNs), or your data varies over time which would require periodic retraining (especially, at unpredictable intervals), then DL probably is a mismatch for you.


Does anybody know what would happen if I changed input shape of pytorch models? In this https://pytorch.org/vision/stable/models.html tutorial it clearly states:

All pre-trained models expect input images normalized in the same way, i.e. mini-batches of 3-channel RGB images of shape (3 x H x W), where H and W are expected to be at least 224. The images have to be loaded in to a range of [0, 1] and then normalized using mean = [0.485, 0.456, 0.406] and std = [0.229, 0.224, 0.225].

Does that mean that for example if I want my model to have input size 128x128 it is or if I calculate mean and std which is unique to my dataset that it is gonna perform worse or won't work at all? I know that with tensorflow if you are loading pretrained models there is a specific argument input_shape which you can set according to your needs just like here:
tf.keras.applications.ResNet101(
include_top=True, weights='imagenet', input_tensor=None,
input_shape=None, pooling=None, classes=1000, **kwargs)

I know that I can pass any shape to those (pytorch) pretrained models and it works. What I wanna understand is can I change input shape of those models so that I don't decrease my models training performance?

In all pre-trained models, the input image has to be the same shape; the transform object resizes the image when you add it as a parameter to your dataset object
transform = T.Compose([T.Resize(256), T.CenterCrop(224), T.ToTensor()])
dataset = datasets.ImageNet(".", split="train", transform=transform)

T.Resize(256) changes the image shape then T.CenterCrop(224) takes a random crop to help with overfitting in the training data.
The mean and std are applied to each image before they are used as an input to the model. They were determined when the model was trained, I assume they were hyperparamters.


What should the action space for the card game Crib be? I'm working on creating an environment for a card game, which the agent chooses to discard certain cards in the first phase of the game, and uses the remaining cards to play with. (The game is Crib if you are familiar with it.)

How can I make an action space for these actions? For instance, in this game, we could discard 2 of 6 cards, then choose 1 of 4 remaining cards to play, then 1 of 3 remaining cards, then 1 of 2 remaining cards. How do I model this?

I've read this post on using MultiDiscrete spaces, but I'm not sure how to define this space based on the previous chosen action. Is this even the right approach to be taking?

I've actually implemented this game before using deep reinforcement learning. You are dealing with a dynamic action space here, where the action space may change at each time step of the game (or more generally the MDP). First, let's discuss the actual action spaces in each one of the two phases of Crib (or Cribbage) and formalize the question.

Phase 1: The Discard:In this phase, you are concurrently discarding  2 cards without respect to order. Therefore, you have a fixed discrete action space of size ${{6}\choose{2}} = 15$.

Phase 2: The Play: In this phase, you and your opponent are sequentially playing one of each of your remaining 4 cards (the original 6 minus the 2 discarded from phase 1). Therefore, you have a discrete action space of size $4! = 24$. Here's the catch - not every one of these actions is legal. The current state of the game restricts which cards you are allowed to play (the sum of all cards currently played must not be greater than 31). Since you do not know your opponent's cards and/or policy, you do not know which of these 24 actions are valid. To remedy this, the action space should dictate which of your remaining cards may be played at the current time step. Thus, you have a dynamic discrete action space of size 1, 2, 3, or 4 at each time step.

  How can I make an action space for these actions?

Since you didn't specify any implementation standard (e.g. OpenAI Gym), there are multiple paths to take, and they usually depend on your implementation of the state feature vector. In my own implementation, I tinkered with two possible state representations, which are fairly simple to describe.

Possibility 1: Separate State Representations for each Phase: In phase 1, you need to know the cards in your hand and the scores; i.e., the state feature vector could be encoded as a list of [card0, card1, card2, card3, card4, card5, your score, opponent score]. This state represents all information known about the game during phase 1 from a single player's viewpoint; after each time step, the current player may change, and the state must be updated according to the current player's viewpoint. Each card can be encoded as an integer from 1 to 52 (not starting from 0, as we will see in the next paragraph), and the score is an integer from 0 to 120 (tip: sort your cards for a reduced state space and faster convergence). Your action space can be the set of integers from 0 to 14 that maps to a 2-card combination in your hand. Alternatively, you could have a dynamic action space that sequentially asks for 1 of your 6 cards to discard and then for 1 of your remaining 5 cards to discard. The action space can be a subset of integers from 0 to 5 that maps to a single card in your hand. Be careful here - when choosing the second card to discard, your algorithm must know which card was discarded first. You can solve this by adding another component to your state vector that represents the first card that was discarded (set to 0 at the beginning of the phase), and therefore, make sure to update the state after the first discard.

In phase 2, you need to know the 4 cards in your hand, the cut card, the scores, and the cards currently played. Another helpful but unnecessary feature for learning is the current sum of played cards. A possible representation is a list of [card0, card1, card2, card3, card played0, card played 1, card played 2, …,  card played 7, your score, opponent score, cut card, current sum of played cards]. The values of played cards should be initialized to 0 at the beginning of the phase. The state can be updated so that any card in your hand that you have played is set to 0, and any card played by your opponent can be set to its negative value. This will correctly encode which cards have been played from your hand, which cards played are your from your opponent, and all other available information. Consequently, the action space is dynamic and is a subset of integers from 0 to 3 that maps to a single card in your hand.

Possibility 2: Identical State Representations for each Phase Another possibility is to have all of the above information for each phase encoded into a single state representation along with the phase number. State features that are unique to phase 2 that aren't relevant for phase 1 can be set to 0 during phase 1 and vice versa. With this representation, the state feature vector is of the same length at all time steps of the game, and the action space will change as described above. The important ideas for the encoding are exactly the same as above and will change based on your particular implementation, so I won't include the details here.

  How do I model this?

If you are going to implement something similar to Possibility 1, then you may need two agents that each learn a policy for a separate phase. For starters, you could use Q-learning or DQN and take the action with greatest q-value at each timestep, making sure that the chosen action is always a member of the current action space.

If you are going to implement something similar to Possibility 2, then you may only need a single agent that learns a policy for each phase, simply because the phase is a feature of the state. Essentially you are trading off a more complicated state representation for a simpler learning algorithm.

  I've read this post on using MultiDiscrete spaces, but I'm not sure how to define this space based on the previous chosen action. Is this even the right approach to be taking?

After reading the OpenAI Gym documentation, it looks like the MultiDiscrete space is a product of Discrete spaces. Therefore, it is a fixed action space and inherently not what you want here (a dynamic action space). I don't believe that OpenAI Gym standards will support dynamic action spaces natively. You would need to do some extra work such as providing a method that returns the current action space of the environment. Alternatively, if you want to follow the (state, reward, done, info) signal paradigm from OpenAI Gym, you may provide the current action space in the info dictionary. Finally, another idea is to allow the agent to always choose an action from a larger fixed action space (e.g. the set of integers from 0 to 3 for phase 2) and then penalize the agent through the reward signal whenever it chooses an action that is not a member of the current action space (e.g. if the chosen card was already played in phase 2). Afterward, you would return the current game state as the next state and make the agent try again.

My advice is to first determine the state representation, and the rest of your implementation should follow, using the ideas above.


Why is the policy not a part of the MDP definition? I'm reading an article on reinforcement learning, and I don't understand why the agent's policy $\pi$ is not part of definition of Markov Decision process(MDP):

Bu, Lucian, Robert Babu, and Bart De Schutter. "A comprehensive survey of multiagent reinforcement learning." IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 38.2 (2008): 156-172.

My question is:

Why the policy is not a part of the MDP definition?

The MDP defines the environment (which corresponds to the task that you need to solve), so it defines e.g. the states of the environment, the actions that you can take in those states, the probabilities of transitioning from one state to the other and the probabilities of getting a reward when you take a certain action in a certain state.

The policy corresponds to a strategy that the RL agent can follow to act in that environment. Note that the MDP doesn't define what the agent does in each state. That's why you need the policy! An optimal policy for a specific MDP corresponds to the strategy that, if followed, is guaranteed to give you the highest amount of reward in that environment. However, there are multiple strategies, most of them are not optimal. This should clarify why the policy is not part of the definition of the MDP.


What are the steps that I need to follow to build a neural network for face recognition? I have developed face recognition algorithms by using pre-built libraries in Python and open CV. However, suppose if I want to make my own neural network algorithm for face recognition, what are the steps that I need to follow?
I have just seen Andrew Ng's course videos (specifically, I watched 70 videos).

My implementation to increase detection on video is using object tracking algorithms.
More specifically, first, I detect the object using a trained classifier. Then I track the object with the KCF algorithm. If the object tracker misses the object, again I call for the classifier.


Do specific units exists for measuring the intelligence of a machine? We can measure the power of the machine with the number of operation per second or the frequency of the processor. But does units similar of IQ for humans exist for a AI?
I'm asking for a unit which can give countable result so something different from a Turing Test which only give a binary result.

One of the challenges of AI is defining Intelligence.
If we could precisely define general intelligence then we could program it into a computer. After all an algorithm is a process so well defined that it can be run on a computer.

Narrow AI can be evaluated on its success at achieving goals in an environment. In domains such as computer vision and speech recognition narrow AI algorithms can be easily evaluated.

Many universities curate narrow AI tests. Fei-Fei Li a professor at Stanford who directs the Artificial Intelligence lab there organises the annual ImageNet Challenge. In 2012 Geoffrey Hinton famously won the competition by building a Deep Neural Network that could recognize pictures more accurately than humans can.

To my knowledge the testers commonly use Precision and recall evaluation metrics


What is the optimal value function of the scaled version of the reward function? Consider the reward function $r(s, a)$ with optimal state-action value function $q_*(s, a)$. What would be the optimal state-action value function of $c r(s, a)$, for $c \in \mathbb{R}$? Would it be $c q_*(s, a)$?

The Bellman optimality equation is given by
$$q_*(s,a) = \sum_{s' \in \mathcal{S}, r \in \mathcal{R}}p(s',r \mid s,a)(r + \gamma \max_{a'\in\mathcal{A}(s')}q_*(s',a')) \tag{1}\label{1}.$$
If the reward is multiplied by a constant $c > 0 \in \mathbb{R}$, then the new optimal action-value function is given by $cq_*(s, a)$.
To prove this, we just need to show that equation \ref{1} holds when the reward is $cr$ and the action-value is $c q_*(s, a)$.
\begin{align}
c q_*(s,a)
&= \sum_{s' \in \mathcal{S}, r \in \mathcal{R}}p(s',r \mid s,a)(c r + \gamma \max_{a'\in\mathcal{A}(s')} c q_*(s',a')) \tag{2}\label{2}
\end{align}
Given that $c > 0$, then $\max_{a'\in\mathcal{A}(s')} c q_*(s',a') = c\max_{a'\in\mathcal{A}(s')}q_*(s',a')$, so $c$ can be taken out of the $\operatorname{max}$ operator. Therefore, the equation \ref{2} becomes
\begin{align}
c q_*(s,a)
&= \sum_{s' \in \mathcal{S}, r \in \mathcal{R}}p(s',r \mid s,a)(c r + \gamma c \max_{a'\in\mathcal{A}(s')} q_*(s',a'))  \\
&= \sum_{s' \in \mathcal{S}, r \in \mathcal{R}}c p(s',r \mid s,a)(r + \gamma \max_{a'\in\mathcal{A}(s')} q_*(s',a')) \\
&= c \sum_{s' \in \mathcal{S}, r \in \mathcal{R}} p(s',r \mid s,a)(r + \gamma \max_{a'\in\mathcal{A}(s')} q_*(s',a')) \\
q_*(s,a) &= \sum_{s' \in \mathcal{S}, r \in \mathcal{R}} p(s',r \mid s,a)(r + \gamma \max_{a'\in\mathcal{A}(s')} q_*(s',a')) \tag{3}\label{3}
\end{align}
which is equal to the the Bellman optimality in \ref{1}, which implies that, when the reward is given by $cr$, $c q_*(s,a)$ is the solution to the Bellman optimality equation. Consequently, in this case, the set of optimal policies does not change.
If $c=0$, then \ref{2} becomes $0=0$, which is true.
If $c < 0$, then $\max_{a'\in\mathcal{A}(s')} c q_*(s',a') = c\min_{a'\in\mathcal{A}(s')}q_*(s',a')$, so equation \ref{3} becomes
\begin{align}
q_*(s,a) &= \sum_{s' \in \mathcal{S}, r \in \mathcal{R}} p(s',r \mid s,a)(r + \gamma \min_{a'\in\mathcal{A}(s')} q_*(s',a'))
\end{align}
which is not equal to the Bellman optimality equation in \ref{1}.


Should the weights of a neural network be updated after each example or at the end of the batch? Should the weights of a neural network be updated after each example or at the end of the batch? Do I need a normalization factor in the second case?

The backpropagation step is generally used to compute the gradients and update the weights.

Let us say, you are implementing gradient descent, select the whole training batch, perform the forward propagation using the current set of neurons/weights to get the classification output. Then compute the loss/cost with respect to the actual labels, (Note: This step contains dividing the loss and divide it by the number of training example to get the cost function). Once you get the cost you backpropagate by computing the gradients at each layer with respect to cost from output to input and perform the weight updates.

If you are using mini-batch, you compute the cost for that mini-batch, divide by the number of examples in the mini-batch, then compute the gradients and perform the update in the backprop step.


Do text compression tests qualify winRar or 7zip as intelligent? I read this paper Text Compression as a Test for Artificial Intelligence, Mahoney, 1999.
So far I understood the following:
Text Compression tests can be used as an alternative to Turing Tests for intelligence.
The Bits per character score obtained from compression of a standard benchmark corpus, can be used as a quantitative measure for intelligence
My questions:

Is my understanding of the topic correct?
Does this mean that applications like 7zip/WinRar are intelligent?
How are the ways a human compresses information (as in form of summary) and ways a computer compresses (using Huffman coding or something) are compatible? How can we compare that?

The paper suggests an alternative test to the famous Turing test, which tests a machine's ability to exhibit intelligent behavior equivalent to, or indistinguishable from, that of a human.
In this test, if winRar or 7zip will compress a file similarly to how a human would compress a file (how does a human compress a file?!), then, yes, those programs will pass the test and will be considered intelligent.

... thus compression ratio on a standard benchmark corpus could be used as an objective and quantitative alternative test for AI (Mahoney, 1999).


Train neural net on another computer? I have a complex neural net that will take forever on my laptop to train and i dont have a computer with a GPU, is there a way to run a python script on another computer without having to install an IDE on that computer? (for instance, if i went to an internet cafe)

Python doesn't require an IDE. You just need to install the Python interpreter to run a Python program.

Running machine learning on another computer is in fact industry standard. Google, Amazon and Azure can provide you a virtual machine, which you can run anything you want.

I personally rent a powerful machine with GPUs on Google to do my machine learning analysis. It's not free but very powerful.


Is Deep Q Neural Network (DQN) applicable only with images as inputs? More precisely: is DQNN applicable only when we have high translational invariance in our input(s)?

Starting from the original paper on nature (here a version stored on googleapis) and after looking online for some other implementation and based on the fact that this NN starts with convolutional layers, I think that is based on the assumption that we feed the network with images but I'm not so sure. In the case that DQNN can be used with other types of inputs, please feel free to include examples in your answer. Also, references will be appreciated.

More precisely: is DQNN applicable only when we have high translational invariance in our input(s)?

No, DQN (that is the commonly-used abbreviation by the way, there also is a dqn tag which you may wish to use) is not restricted to images or other kinds of inputs with those properties, it can be used with pretty much any kinds of inputs.

The DQN algorithm should be viewed separately from the Neural Network Architecture though. DQN can be used with any kind of Neural Network architecture. Yes, the most commonly-used type of architecture used with DQN is probably architectures that start with a bunch of convolutional layers, and those kinds of architectures are best suited for image-based inputs. This is not a requirement though. If you have other kinds of features where convolutional layers don't make a lot of sense, you can, for example, simply start directly with some ReLU layers.

Here are two examples of DQN being used for problems without image-based inputs from the OpenAI baselines repository:

DQN + Mountain Car
DQN + Cartpole

In both cases, they're using deepq.models.mlp() to construct a relatively straightforward Multi-Layered Perceptron architecture, without any convolutional layers. In this example for Atari games, they do have image-based input and therefore also construct an architecture with convolutional layers using deepq.models.cnn_to_mlp().

Note that, if you have relatively straightforward features, it may often not be necessary to use Deep RL approaches; tabular RL approaches of RL with Linear Function Approximation may work just as well in cases where you already have good features. If you have image-based inputs, those kinds of approaches are much less likely to work well. The disadvantage of Deep RL approaches like DQN is that they tend to require much more experience / data than simpler approaches.

So, generally the interesting question isn't "can DQN (or another Deep RL approach) handle my inputs?", because the answer is probably yes. The more important question would often be "do I have to use DQN (or another Deep RL approach?". The answer to that question will almost always be yes if you have image-based inputs, but relatively often be no if you already have good features as inputs.

Another class of problems where Deep RL is really popular nowadays is continuous control problems (e.g. robot simulators, MuJoCo, etc.). These don't have image-based inputs, but still generally require Deep RL (not DQN though; DQN doesn't handle continuous-valued outputs very well, it generates discrete outputs).


Being confused of distribution notations in Deep Learning book In chapter 5 of Deep Learning book of Ian Goodfellow, some notations in the
loss function as below make me really confused.

I tried to understand $x,y \sim p_{data}$ means a sample $(x, y)$ sampled from original dataset distribution (or $y$ is the ground truth label).
The loss function in formula 5.101 seems to be correct for my understanding. Actually, the formula 5.101 is derived from 5.100 by adding the regularization.

Therefore, the notation $x,y \sim \hat{p}_{data}$ in formula 5.96 and 5.100 is really confusing to me whether the loss function is defined correctly (kinda typo error or not). If not so, could you help me to refactor the meaning of two notations, are they similar and correct?

Many thanks for your help.

At page 130 of the same book, the author states that $\hat{p}_\text{data}$ is an empirical distribution defined by the training data. Similarly, at page 129, he states that $p_\text{data}$ is the true distribution that generates the set $\mathbb{X} = \{ \boldsymbol{x}^{(1)}, \dots, \boldsymbol{x}^{(m)} \}$.

What is the difference between $\hat{p}_\text{data}$ and $p_\text{data}$? You can think of $\hat{p}_\text{data}$ as a histogram that is calculated from the set $\mathbb{X}$ and $p_\text{data}$ as the true distribution from which the elements in $\mathbb{X}$ are drawn.

The subscript ${\boldsymbol{x}, y \sim \hat{p}_\text{data}}$ in the expectation $\mathbb{E}_{\boldsymbol{x}, y \sim \hat{p}_\text{data}}$ indicates that the expectation is taken with respect to the samples drawn from the empirical distribution $\hat{p}_\text{data}$. In other words, you will optimise the objective function $J$ using the training data. Have a look at this question for more info.

The subscript ${\boldsymbol{x}, y \sim p_\text{data}}$ in the expectation of formula $5.101$ is a typo. In fact, in this online version of the book, at page 151, the subscript of the expectation is  ${\boldsymbol{x}, y \sim \hat{p}_\text{data}}$.


Is this dataset with only two features suitable for clustering with k-means? I am working with the K-means clustering algorithm for unsupervised learning.

Is the following dataset suitable for the k-means clustering task or not? Why or why not? The dataset has only two features.

One problem with clustering algorithms is that they will typically find you a solution, ie they will split your data set into clusters, but it will find you a structure even if there isn't one. Your data looks like it could consist of about 5 to 7 clusters, but it could equally well just be 2 or only 1.

What you need to do after the clustering is to assess the quality of the result. I recommend having a look at Finding Groups in Data by Kaufman & Rousseeuw. They discuss various clustering algorithms and also a procedure that works out how cohesive your clusters are. Though it is 30 years old, it is an excellent book on the topic.

You also have the issue of choosing a value for k in your clustering: I usually start with two, and increase it from there; at each step I compute the cohesion of the result using their method, until I get the best score. This is an objective way of finding a good value for k and usually a reasonable clustering result.

The ultimate test, of course, is then if looking at the result makes sense to you. No cluster algorithm can do that for you.


Video engagement analysis with deep learning I am trying to rank video scenes/frames based on how appealing they are for a viewer. Basically, how "interesting" or "attractive" a scene inside a video can be for a viewer. My final goal is to generate say a 10-second short summary given a video as input, such as those seen on Youtube when you hover your mouse on a video.

I previously asked a similar question here. But the "aesthetics" model is good for ranking artistic images, not good for frames of videos. So it was failing. I need a score based on "engagement for general audience". Basically, which scenes/frames of video will drive more clicks, likes, and shares when selected as a thumbnail.

Do we have an available deep-learning model or a prototype doing that? A ready-to-use prototype/model that I can test as opposed to a paper that I need to implement myself. Paper is fine as long as the code is open-source. I'm new and can't yet write a code given a paper.

One of the key terms in the literature that you are looking for is video captioning.

You can have a look at some of the relevant papers with code on this subject. In short, it is an active area of research and a difficult problem, one reason is because videos are still difficult to learn about (because of larger amount of data + larger model, etc...) and this model has to be working with two modalities of data: text and image.

A paper that you might want to start with is Deep Visual-Semantic Alignments for Generating Image Descriptions which works on single images. In short, you can use something similar like in the paper: object detector (e.g. Faster RCNN) to extract visual features and feed them into the state of an RNN (LSTM) which would output a sequence of words in your summary (see picture below).


How can I implement 2D CNN filter with channelwise-bound kernel weights? I would like to bind kernel parameters through channels/feature-maps for each filter. In a conv2d operation, each filter consists of HxWxC parameters I would like to have filters that have HxW parameters, but the same (HxWxC) form.
The scenario I have is that I have 4 gray pictures of bulb samples (yielding similar images from each side), which I overlay as channels, but a possible failure that needs to be detected might only appear on one side (a bulb has 4 images and a single classification). The rotation of the object when the picture is taken is arbitrary. Now I solve this by shuffling the channels at training, but it would be more efficient if I could just bind the kernel parameters. Pytorch and Tensorflow solutions are both welcome.

Assuming you want HxWx1 kernel to perform convolution on hxwxc images.
Here's sample code which uses single channel kernel to operate on multichannel feature: maps
import torch
import torch.nn as nn
import torch.nn.functional as F

class model(nn.Module):
    def __init__(self, in_ch=4):
        super().__init__()
        self.in_ch  = in_ch
        # single channel kernel initialization
        self.kernel = nn.Conv2d(in_channels=1, out_channels=1, kernel_size=1, stride=1)

    def forward(self, x):
        bs, ch, w, h = x.shape
        x   = x.view((bs, ch, 1, w, h))
        out = self.kernel(x[:, 0])
        # reusing of same kernel
        for i in range(1, self.in_ch):
            out = torch.cat((out, self.kernel(x[:, i])), 1)
        return out

net = model(4)
print(net)
inp = torch.randn((10, 4, 100, 100))
out = net(inp)
print(out.shape)

(The main hack is in the forward function)


Is it practical to train AlphaZero or MuZero (for indie games) on a personal computer? Is it practical/affordable to train an AlphaZero/MuZero engine using a residential gaming PC, or would it take thousands of years of training for the AI to learn enough to challenge humans?
I'm having trouble wrapping my head around how much computing power '4 hours of Google DeepMind training' equates to my residential computer running 24/7 trying to build a trained AI.
Basically, are AlphaZero or MuZero practical for indie board games that want a state of the art AI, or is it too expensive to train?

The vast majority of neural networks are now trained on graphics processing units (GPUs) or specialised accelerator hardware such as tensor processing units (TPUs).
In Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm, Silver et al. say that the training process involved 5,000 first-generation TPUs generating self-play games and 64 second-generation TPUs for training. This is certainly far beyond what any practical gaming computer is likely to achieve, as you'll likely only have one GPU, and that might not even rival a single TPU. Training on the CPU will be substantially slower again than either a GPU or TPU. Training would be orders of magnitude slower; you might find these benchmarks by Wang et al. of interest.