How does the target network in double DQNs find the maximum Q value for each action? I understand the fact that the neural network is used to take the states as inputs and it outputs the Q-value for state-action pairs. However, in order to compute this and update its weights, we need to calculate the maximum Q-value for the next state $s'$. In order to get that, in the DDQN case, we input that next state $s'$ in the target network.
What I'm not clear on is: how do we train this target network itself that will help us train the other NN? What is its cost function?

Both in DQN and in DDQN, the target network starts as an exact copy of the Q-network, that has the same weights, layers, input and output dimensions, etc., as the Q-network.
The main idea of the DQN agent is that the Q-network predicts the Q-values of actions from a given state and selects the maximum of them and uses the mean squared error (MSE) as its cost/loss function. That is, it performs gradient descent steps on
$$\left(Y_{t}^{\mathrm{DQN}} -Q\left(s_t, a_t;\boldsymbol{\theta}\right)\right)^2,$$
where the target $Y_{t}^{\mathrm{DQN}}$ is defined (in the case of DQN) as
$$
Y_{t}^{\mathrm{DQN}} \equiv R_{t+1}+\gamma \max _{a} Q\left(S_{t+1}, a ; \boldsymbol{\theta}_{t}^{-}\right)
$$
$\boldsymbol{\theta}$ are the Q-network weights and $\boldsymbol{\theta^-}$ are the target network weights.
After a usually fixed number of timesteps, the target network updates its weights by copying the weights of the Q-network. So, basically, the target network never performs a feed-forward training phase and, thus, ignores a cost function.
In the case of DDQN, the target is defined as
$$
Y_{t}^{\text {DDQN}} \equiv R_{t+1}+\gamma Q\left(S_{t+1}, \underset{a}{\operatorname{argmax}} Q\left(S_{t+1}, a ; \boldsymbol{\theta}_{t}\right) ; \boldsymbol{\theta}_{t}^{-}\right)
$$
This target is used to decouple the selection of the action (i.e. the argmax part) from its evaluation (i.e. the computation of the Q value at the next state with this selected action), as stated the paper that introduced the DDQN)

The max operator in standard Q-learning and DQN, in (2) and (3), uses the same values both to select and to evaluate an action. This makes it more likely to select overestimated values, resulting in overoptimistic value estimates. To prevent this, we can decouple the selection from the evaluation


Which AI Technique best suited to discovering non-linear relationships in data? I am interested in exploring whether AI techniques can derive hidden patterns of relationships in a data set.  For example, from among house size, lot size, age of house and asking price,   what formula best predicts selling price?

In explorations around how this might be done, I tried to use a neural network to solve for a predictable relationship between two variables to predict a third, so I trained my neural network with inputs consisting of the length of two sides of a triangle, and the result being the length of the hypotenuse.  It couldn't get it to work.

I was told by somebody who understands all this better than me that the reason it failed is because conventional neural networks are not good at modeling non-linear relationships.

If that is true, I wonder if there is some other AI technique that could 'derive' a network modeling the phythagorean theorem from a training data set with better results than a normal neural network?

You are mixing up lots of things here. Specifically, you seem to be lacking a basic understanding of artificial neural networks and what they can do (e.g. what type of articifial neural networks are linear classifiers/regressors and which can model non-linear relationships).

Therefore, I'd take a step back and start with understanding the basics of AI. The go-to book for that is 'Artificial Intelligence: A Modern Approach' by Russel and Norvig. It might be a slower (and more theoretical) start but IMO that is the right approach to actually understand what you are doing.


Are there deep neural networks that have inputs connected with deeper hidden layers? Are there any architectures of deep neural networks that connect input neurons not only with the first hidden layer but also with deeper ones (red lines on the picture)?

If so could you give some names or links to research papers?

This type of connections are called skip or residual connections. There are numerous works which employs this type of mechanism, for example: ResNet, SkipRNN. In addition here you can find a paper that empirically explores the skip connections for sequential tagging, or this one for speech enhancement.


Can a fully convolutional network always return an image of the same size as the original? I'm trying to perform a segmentation task on images of multiple sizes using fully convolutional neural networks.

Currently, I'm using EfficientNet as a feature extractor, and adding a deconvolution/backwards convolution/transposed convolution layer as described in the original Fully Convolutional Networks for Semantic Segmentation paper.

But this transposed convolution layer doesn't return a filter of a size equivalent to the original image for images of varying sizes.

For example, let's assume the original image is $100 \times 100$, and the last layer contains filters of size $50 \times 50$. To get a filter of the same size as the original, you would need a transposed convolution layer of size $51 \times 51$.

Now, assume you passed in an image of size $200 \times 200$. The last layer would contain filters of size $100 \times 100$. That same transposed convolutional filter of size $51 \times 51$ would result in an output of size $150 \times 150$.

Is there any way to make it so that a fully convolutional network always returns an image of the same size as the original?

I ended up using a work around.
I set up the network so that an C x C (i.e. 320 x 320) input would output a C x C mask for some constant C (in my case it was 320).
I then resized the image I wanted to pass in to C x C, and then resized the output back to the original size of the Image.


Approaches to poker tournament winner prediction? I’ve  done my research and could not find answer anywhere else. My apologies in advance if same problem is answered in different terms on stack-overflow.

I am trying to solve poker tournament winner prediction problem. I’ve millions of historical records in this format:

Players ==> Winner
P1,P2,P4,P8 ==> P2
P4,P7,P6 ==> P4
P6,P3,P2,P1 ==> P1

What are some of the most suitable algorithms to predict winner from set of players.

So far I have tried decision trees, XGboost without much of a success.

This looks like it maps very nicely onto Association Mining. In association mining, you are trying to find items from a discrete set that often co-occur in transactions. For instance, you might want to find the items that most commonly appear in an online shopping cart together.

In your case, the problem amounts to:

Split up the data into sub-problems by who won.
Perform association mining on the sets of players in each sub-problem, using, e.g. the apriori algorithm.

The resulting rules will then have associated confidence factors. When you want to predict who will win, you can take the winner suggested by the rule with the greatest confidence.

The other approach suggested by your problem is to model it as classification. Here you are trying to assign labels (who won) to input vectors (who played). The process would look like:

Map the sets of who played into binary features. So the subset P1,P2, P4,P8 might be represented with {1,1,0,1,0,0,0,1} if there are only 8 players. Map the winning player onto a numeric class (e.g. the numbers 1-8).
Run any classification algorithm to create a model that predicts the class from the binary features. A decision tree learner might be a interesting starting point if you want to understand which factors are important to the model. There are many other techniques however.

You can use the model you train to make predictions about future games with the same set of players.


Can we use NLP to understand/parse/compile programming code? I wonder if we can use Natural Language Processing (NLP) to process programming code:
Given a piece of code, can we

Translate it to human language to understand what it does? The input could be a function definition（normally lack of documentation) in Python and the output could be the documentation for that function.
Compile or translate it to another programming language? Compile Python code to C or machine code, or translate C code to Python code?

Yes, and that is the ambition of the Decoder project (H2020 funded in Europe). Analyzing with NLP techniques the comments in e.g. C or C++ source code (of open source projects, and perhaps their git logs).
I even happen to be on the photo.
My dream is to try similar things, in a few years, in RefPerSys. You could join that project, BTW.


Is my approach to building an RNN to predict the probability that the word is in English appropriate? Goal
To build an RNN which would receive a word as an input, and output the probability that the word is in English (or at least would be English sounding).
Example
input:  hello
output: 100%

input:  nmnmn
output: 0%

Approach
Here is my approach.
RNN
I have built an RNN with the following specifications: (the subscript $i$ means a specific time step)
The vectors (neurons):
$$
x_i \in \mathbb{R}^n \\
s_i \in \mathbb{R}^m \\
h_i \in \mathbb{R}^m \\
b_i \in \mathbb{R}^n \\
y_i \in \mathbb{R}^n \\
$$
The matrices (weights):
$$
U \in \mathbb{R}^{m \times n} \\
W \in \mathbb{R}^{m \times m} \\
V \in \mathbb{R}^{n \times m} \\
$$
This is how each time step is being fed forward:
$$
y_i = softmax(b_i) \\
b_i = V h_i \\
h_i = f(s_i) \\
s_i = U x_i + W h_{i-1} \\
$$
Note that the $ + W h_{i-1}$ will not be used on the first layer.
Losses
Then, for the loss of each layer, I used cross entropy ($t_i$ is the target, or expected output at time $i$):
$$
L_i = -\sum_{j=1}^{n} t_{i,j} \ln(y_{i,j})
$$
Then, the total loss of the network:
$$
L = \sum L_i
$$
RNN diagram
Here is a picture of the network that I drew:

Data pre-processing
Here is how data is fed into the network:
Each word is split into characters, and every character is split into a one-hot vector. Two special tokens START and END are being appended to the word from the beginning and the end. Then the input at each time step will be every sequential character without END, and the output at each time step will be the following character to the input.
Example
Here is an example:

Start with a word: "cat"
Split it into characters and append the special tags: START  c  a  t  END
Transform into one-hot vectors: $v_1, v_2, v_3, v_4, v_5$
Then the input is $v_1, v_2, v_3, v_4$ and the output $v_2, v_3, v_4, v_5$

Dataset
For the dataset, I used a list of English words.
Since I am working with English characters, the size of the input and output is $n=26+2=28$ (the $+2$ is for the extra START and END tags).
Hyper-parameters
Here are some more specifications:

Hidden size: $m=100$
Learning rate:  $0.001$
Number of training cycles:  $15000$ (each cycle is a loss calculation and backpropagation of a random word)
Activation function: $f(x) = \tanh(x)$

Problem/question
However, when I run my model, I get that the probability of some word being valid is about 0.9 regardless of the input.
For the probability of a word begin valid, I used the value at the last layer of the RNN at the position of END tag after feeding forward the word.
I wrote a gradient checking algorithm and the gradients seem to check up.
Is there conceptually something wrong with my neural network?
I played a bit with $m$, the learning rate, and the number of cycles, but nothing really improved the performance.

while using a neural network for this type of problem is not the ideal use-case, it is a good exercise.
In terms of conceptual issues, the most concerning that I see is the loss: $\sum_{i=1}^N L_i$.
First issue, is that it validates loss at each time step equivalently. This is probably not ideal because in the example (cat), we dont expect it to know its English from just c or ca, but at cat. The quickest fix and probably the best, is to just use $L = L_N$. Though an argument would be that the model should become more and more aware as it gets more letters, and this loss function doesnt achieve that, so another solution would be to add another fixed parameter that you can play with: $L = \sum_i r^{-i}L_i$ where $0 \lt r \lt 1$. Note that $r^{-i}$ can be replaced with any function that increases in size.
Another issue with the loss is it doesnt normalize, meaning on average, larger words will hold more weight to the model than smaller ones, while this may be intended it should be noted and considered (also note if you do end up going with $L=L_N$ this will no longer be a concern.
Hope this helps


How does an unsupervised learning model learn? How does an unsupervised learning model learn, if it does not involve any target values?

Supervised learning is typically an attempt to learn a mathematical function, $f(\bf X)=\bf y$. For this, you need both the input vector $\bf X$ and the output vector $\bf y$. The model outputs have whatever dimensionality that the target values have.

Unsupervised learning models instead learn a structure from the data. A clustering model, for example, is learning both how many clusters exist in the data (a number that's not the same type as the inputs) and where those clusters are located (which is also a different type from the inputs). The output of running this model on a new datapoint $x$ is not the same type as $x$, but instead a classification label.

Similarly, time series models learn parameters that symbolize how vectors in the input relate to each other, rather than raw inputs themselves.

As for how they learn, the structures are mathematical objects whose fitness is determined by the input data. The simplest possible unstructured unsupervised learning problem is probably "what's the mean of the data?", and it should be clear how that's 'learned' through processing the input. More sophisticated models are just adding more pieces to that calculation.


Can I do topic classification of Arabic text (software requirements) without a training dataset? I am trying to make a text classification for Arabic data. The problem is that there is no labeled Arabic dataset for this data. My question is then: is possible to do a classification without a training dataset? If yes, what methods can I use?

Yes, it is possible to do it. This can be posed as an unsupervised text classification. You can look at TF-IDF, Neural networks (BERT) etc. for creating embedding of your text and then, use clustering techniques like K-Means, KNN etc. for classification.


Are winning and imitating the primary indicators of intelligence? Conjecture 1: The smartest chess playing system is the one that wins the tournament.

Conjecture 2: The computer system that imitates human dialog is as smart as the human whose dialog was imitated.

These are legacy views from the twentieth century.

Are they correct?

Response to Comments

The question is not about agents. It is about indicators, or metrics if you wish. The question is whether the established game championship and Turing's thought experiment, the dialog centered imitation game, are correctly designed metrics to establish the extent of intellectual ability.

That someone voted to close this question on the basis of broadness is irrelevant. This question and ones like it are important questions for AI researchers and enthusiasts to ask about the set of capabilities we call intelligence, which we attempt to realize in artificial systems.

There may also be a tendency to think of questions like this as seeking an opinion. It is certainly true that our opinions are not useful to the furtherance of AI, and opinion is not what this question invite when it asks, "Is this [pair of conjectures that guides much AI research] correct?"

Correctness must draw on mathematical rigor leading to a yes or a no. Some reasoning leading to the conclusion of correctness or incorrectness must be given. The answer may be yes in one context and no in another, but this is not because of the broadness of the question but because of the broadness of the many useless opinions and possibly counterproductive conjecture that remains from early thinking about what intelligence is and how to gauge its presence in computing machinery or in people.

Questions about intelligence from this author are intended to narrow the terms and concepts so that they can be expressed mathematically. Only with a mathematically terse and elegant set of fundamentals can AI progress as a field in the sciences. Some interesting opinion will probably need to be discarded and a logically constructed consensus must replace broadness and ambiguity for this to occur.

Winning

My feeling is that utility is the standard of determining intelligence, and merely connotes the strength of a decision making algorithm.

Winning is a metric involving strength in relation to other agents, but may not always result in optimal global utility.  (Here I'm thinking of a game AI that always wins.  In a game like Chess, a subset of serious players may appreciate this, but it's unclear that this would be any kind of fun for the average player.  Similarly, an unbeatable AI in a game like Civilization or Stellaris would be very poorly received by most players.)

In the paper Depth In Strategic Games they formalize the term "strategy ladder". A simple definition of this concept is a series of AIs that are "meaningfully weak".  Specifically, players learn to beat them and in so doing learn different levels of strategy, ascending up ladder.

In this example, user engagement is the ultimate goal, not the AI winning every game!

Imitation

Turing makes a compelling case that imitation is an indication of intelligence (self-awareness and "true comprehension" as strictly corollary.)

I like to reduce this idea to Captcha, because it involves utility.  At some point Captcha may no longer work because the average bot is a "smart" in solving them as the lowest-common denominator human.

The same goes for chat bot--it they can successfully imitate humans, they can be considered as smart as those humans in regard to chatting.

Conclusion: Both winning and imitation are indicators of intelligence, whether machine-based of biological.


Is it ethical to implement self-defence for street walking AI robots? Would it be ethical to implement AI for self-defence for public walking robots which are exposed to dangers such as violence and crime such as robbery (of parts), damage or abduction?

What would be pros and cons of such AI behavior? Is it realistic, or it won't be taken into account for some obvious reasons?

Like pushing back somebody when somebody start pushing it first (AI will say: he pushed me first), or running away on crowded street in case algorithm will detect risk of abduction.

The question mentions "walking robot", but it may be illustrative to re-frame the discussion in terms of self-driving cars, because:

It gives a common point of reference, rather than everyone having their own separate vision of how vulnerable/powerful a kung-fu walking robot might be.
We already know a lot about societal attitudes to car theft.
Given that autonomous vehicles will soon be mainstream, the morality of the question is then more of a pressing issue.

So, should a self-driving car run someone over (likely killing them) if they try to steal it? I'm hoping that few people would argue that it should.

Should it attempt to do a lesser amount of damage (say, calculated to hopefully only break a leg)?

Again, I'd argue not. The main reason for saying this is that our decision-making algorithms are simply not sufficiently context aware to be able to decide whether theft or harm is the intent. To concretely illustrate this: a recent fatality arose because a self-driving Tesla was oblivious to context to the extent that it couldn't distinguish between a high-sided van and empty space.

Under those circumstances, it's probably best not to allow commercial autonomous systems to cause physical damage (even to inanimate objects).

'Running away' (or rather, 'driving away', in the case of the car) is another matter: driving is what it's designed to do.


Recent methods for Decision Support System (DSS) In Decision Support System (DSS), we rank items based on predetermined weighted criteria. For example, we want to rank prospective programmers based on their working experience, required salary, set of skills, age, etc. We rank using weights for each criterion that we have previously defined. The simplest method is using Simple Additive Weighting (SAW).
As far as I know, DSS is included in knowledge-based AI (it's a mandatory subject in AI specialization in most universities in my country).
My question:

With the development of AI/ML/DL today, is there another modern approach that can be used to solve similar problems?

At first, I thought it's similar with Content-Based Recommender System, but it looks different as we don't have "user" in DSS.

In my opinion, the sucess level of the Deep Learning models is perfectly suited for DSS. For example, if you have data you can built perfect Deep Reinforcement Model to rank programmers. In general AI models, specifically Deep models, are not mature enough to use without supervision of human being. But they are very suitable to build support systems.


Is robotic process automation related to AI? I am trying to understand if robotic process automation is a field which requires expertise in machine learning.
Does the algorithm behind rpa use machine learning expect from OCR?

In the context of IT systems, "Robotic Process Automation" (RPA) is a term often used to describe a technique where software systems are integrated or work processes are automated through the existing user interface of the applications rather than writing new software to provide integration points.

In that context, RPA has nothing to do with AI or machine learning. In most cases, it does not even require OCR.

For an example of a common use case, let's say you have an old mainframe IT systems for tracking your subscriptions and a new website to let people order subscriptions from their phone.

In this case you might create an "RPA" job that opens the list of new subscription requests from the website and for each of them  opens the old application, clicks of the "new subscription" button, click on the "Customer Name" field, pastes the name, clicks "Customer address" and pastes in the address etc.

In some cases, the RPA job will be exposed as a service with an API that can be called by the new application, so it can dump data directly into the old application. The benefit is that it can do this without any changes to the old application.

It is attractive because defining the steps of copy this, click that can often be defined in visual tools by non-programmers very quickly and at much lower cost than setting up a systems integration project to connect the two systems and because integration through the existing user interface does not require any changes to the application.

In this way, it is similar in spirit to how Excel allows non-programmers to automate calculations by writing formulae and thus automating their spreadsheets.

You will often see RPA proponents putting some AI buzzwords into their presentations but from what I have seen in industry RPA is mostly just a visual scripting technique that is easy to learn and easy to apply.


How to use AI for language recognition? Given an audio track, I'm trying to find a way to recognize the audio language. Only within a small set (e.g. English vs Spanish). Is there a simple solution to detect the language in a speech?

Google has an API you can use. https://cloud.google.com/translate/. Their API can translate audio to text. They also have an API for converting speech to text. The language detection feature should let you detect the language in the resulting text. They have client libraries for the most popular programming languages.


Once self-driving cars are perfected, will they replace manually driven cars? After self-driving cars are perfected to the degree that accident statistics are lower for them than human driven cars, will they replace cars with human drivers?  Will there eventually only be self-driving cars?

It likely to be happen, because it's more convenient that way. In general people, organizations and government are always keen to make things more efficient by standarizing things (computers, technology, law, science, etc.) in order to make it manageable and predictable to reduce the time and minimalize the risk of the same mistakes.

The whole world now moves into technological advancement where automation of everything is where we are going, so we can manage complexities in more reliable way, so we can focus on much bigger picture. This includes technology such as mobiles, computers, UAV (delivery drones), robots and now self-driving cars.

The pros of that change would be:

To have safer streets by introducing autonomous cars on the road.
To have fewer drunk, tired, drugged or crazy drivers.
To avoid poor weather conditions.
To reduce braking distances by dropping driver's reaction time and predicting dangerous situations much earlier.

Source: Cyber Physics
Reducing car deaths and costs of GNP.

  An estimated 1.3 million people die on the world's roads every year with around 50 million injured or disabled by accidents, with accidents costing countries up to four per cent of their Gross National Product (GNP) yearly. - UN News Centre

To have central point of safety improvements, you cannot change people, but you can fix the known safety issue on global scale.
To introduce global standards from the central point (e.g. new law to which manufactures needs to apply).
To increase car safety in general on larger scale.
And so on.

Why we need the 'only self-driving cars'?

The Secretary-General said the UN would work hard to prevent further deaths on the roads:

  Many tragedies can be avoided through a set of proven, simple measures that benefit not only individuals and families but society at large.

Here are my points:

To achieve 'a set of proven measures' - do not allow people to drive - simple.
People tend to break the rules, always, so do not allow them to drive without permission.
Reduce stealing cars and other crime.
Law enforcement dream is to able to stop any car on demand.
Do not allow drunk people to drive a car.
Disallow terrorist attacks, like in Nice where truck killed over 80 people.
Avoid bank robberies and similar which are possible by escaping fast cars.

Is it possible? I believe it depends on specific countries and unions and how quickly we're able to advance and be ready for such change.

To support above points and summarize the 'only self-driving cars' point, please see below references which shows that this is already happening:

2014: Uber will eventually replace all its drivers with self-driving cars
2016: Google's self-driving car system has been officially recognised as a driver in the US.

  The move is seen as a first step towards changing the law for cars that have "no need for a human driver".

2016: Beverly Hills to replace public transport with self-driving cars
2016: San Francisco pitches $149 million plan to replace cars with self-driving vehicles

  San Francisco’s future is autonomous and shared vehicles – and that future may be only a decade away.

2016: Otto Self-Driving Truck Company Wants to Replace Teamsters


Is a reward given at every step or only given when the RL agent fails or succeeds? In reinforcement learning, an agent can receive a positive reward for correct actions and a negative reward for wrong actions, but does the agent also receive rewards for every other step/action?

In reinforcement learning (RL), an immediate reward value must be returned after each action, along with the next state. This value can be zero though, which will have no direct impact on optimality or setting goals.
Unless you are modifying the reward scheme to try and make an environment easier to learn (called reward shaping), then you should be aiming for a "natural" reward scheme. That means granting reward based directly on the goals of the agent.
Common reward schemes might include:

+1 for winning a game or reaching a goal state granted only at the end of an episode, whilst all other steps have a reward of zero. You might also see 0 for a draw and -1 for losing a game.

-1 per time step, when the goal is to solve a problem in minimum time steps.

a reward proportional to the amount of something that the agent produces - e.g. energy, money, chemical product, granted on any stop where this product is obtained, zero otherwise. Potentially a negative reward based on something else that the agent consumes in order to produce the product, e.g. fuel.


Deep audio fingerprinting for word search Simply speaking, I'm trying to somehow search an audio clip for a list of words, and if found, I mark the time stamps. My use-case is profanity check with a list of pre-defined profane words.

Is there any successfull approaches, samples, tools or APIs, possibly through deep learning, to perform this? I'm new to audio processing.

You would need to perform some kind of speech-to-text to get the audio transcription with the corresponding synchronization wrt the audio. Then search in the transcription.

You could use DSAlign by mozilla


What is batch / batch size in neural networks? I have some problems with understanding of the batch concept and batch size. I messed something up. First I start it consider based on convolutional neural network I heard two versions:

When the batch size is set to 50, the first network is fed with 50 images and then learned / recalculated (it doesn't make sense to me, because in this case the network learns one of 50 images).
When the batch size is set to 50, one of 50 neurons is recalculated in the learning process on a single image.

Both of these explanations seem to be wrong to me, so I assume, that I completely don't understand this. What is batch / batch size in RNN? Could you show any example?

I can tell you how I would teach a recurrent neural network. Let's say, that I would like to teach a neural network to predict the weather the next day:

I would take weather data from an expected area from the last 30,000 days.
I would assume that my prediction would be based on measurements from the last 365 days.
I would take data from day 1 to 365 - feed RNN with it and learn.
Then I would take data from day 2 to 366 => feed + learn
Then day 3 to 367 => feed + learn
And so on.

Is this 365 measurement concept a batch size?

You are confusing two concepts, batch size and window size.

Batch Size

The concept of a batch is related to your data and not the number of neurons.  A batch is just a subset of your training data.  The size of the batch affects computational speed, quality, and accuracy.  Large batch sizes will train faster than smaller ones but the model's accuracy can suffer.

There is a rule of thumb that a batch size should be a power of two (e.g. 32, 64, 128, etc.).

Generally speaking larger batch sizes do not generalize as well as smaller batch sizes.  You will need to experiment with the batch size to achieve optimal performance.  Batch size is considered a hyperparameter.

The following video, Batch Size in a Neural Network explained should help clarify things.

Window Size

Window size applies to time series and sequences when using recurrent style networks.  For your case, 365 is the window size.  You did not mentioned how far in the future you are trying to predict.  If we assume that you are predicting one day in the future your output would be the 366th measurement.

Your Application

A batch for you would be a number of training sets.  For example, if we used 32 as a batch size, you would have 32 sets of input windows and outputs.  The first two batches might look like the following:

Batch 1, size 32
1 Data from days 1 to 365 as the input, day 366 as the output
2 Data from days 2 to 366 as the input, day 367 as the output
...
32 Data from days 32 to 396 as the input, day 397 as the output

Batch 2, size 32
1 Data from days 33 to 397 as the input, day 398 as the output
2 Data from days 34 to 398 as the input, day 399 as the output
...


It was not about the model, but about ImageDataGenerator. I found the answer here: https://stackoverflow.com/a/51673998/493080


Wisdom representation? For years I have been dealing with (and teaching) Knowledge Representation and Knowledge Representation languages. I just discovered that in another community (Information Systems and the such) there is something called the "DIKW pyramid" where they add another step after knowledge, namely wisdom.
They define data as being simply symbols, information as being the answer to who/what/when/where?, knowledge as being the answer to how?, and wisdom as being the answer to why?.

My question is: has anyone done the connection between what AI calls data/information/knowledge and these notions from Information Systems? In particular, how would "wisdom" be defined in AI? And since we have KR languages, how would we represent "wisdom" as they define it?

Any references would be welcome…

As with another answer, I am also skeptical of the distinctions made in the DIKW pyramid.
Nonetheless, a very popular machine learning approach for answering 'Why?' questions is the application of Bayesian reasoning: given a causal data model, reverse inference can be used to find the probability distribution of events which lead to a given outcome.
It could be argued that defining 'cause' in terms of distributions rather than specific concrete mechanisms is a rather limited notion of 'Why?'.
However, it may be that there are some forms of causality that we don't know how to represent, specifically 'first-hand experience'. Indeed, common usage of the term 'wisdom' generally refers to first-hand experience, rather than information gained from some other source.
The idea is that knowledge can be expressed declaratively, whereas wisdom must be derived from experience.
For an AI represented as a computer program, the distinction between declarative and first-hand experience might appear irrelevant, since in principle any experience can be encoded and made available without the program having to 'experience' it first-hand.
However, the following humorous definition of `wisdom' might perhaps shed some light on a distinction that's pertinent to AI research:

Knowledge is knowing that a tomato is a fruit.
Wisdom is knowing that you shouldn't eat it with custard.

This notion of 'Wisdom' could be said to require qualia. It is the subject of much debate whether qualia exist and/or are necessary for consciousness - see for example the thought experiment of 'The Black and White Room'.
So the notion is that there is a distinction between having a Bayesian network representation of wisdom that says: "It is 99.7% likely that putting a tomato in custard is undesirable" and the first-hand experience to the effect that it tastes odd with custard.


How does the NEAT speciation algorithm work? I've been reading up on how NEAT (Neuro Evolution of Augmenting Topologies) works and I've got the main idea of it, but one thing that's been bothering me is how you split the different networks into species. I've gone through the algorithm but it doesn't make a lot of sense to me and the paper I read doesn't explain it very well either so if someone could give an explanation of what each component is and what it's doing then that would be great thanks.

The two equations are:

$\delta = \frac{c_{1}E}{N} + \frac{c_{2}D}{N} + c_{3} .\overline{W}$

$f_{i}^{'} = \frac{f_i}{\sum_{j=1}^{n}sh(\delta(i,j))}$

By the way, I can understand the greek symbols so you don't need to explain those to me

The original paper

The first equation deals with distance. Delta, or distance, is the measure of how compatible two genomes are with each other. c1, c2 and c3 are parameters you set to dictate the importance of E,  D and W. Note that if you change cc1, c2 or c3, you will most likely also have to change dt, which is the distance threshold, or the maximum distance apart 2 genomes can be before they are separated into different species. E represents the total number of excess genes. D represents the total number of disjoint genes in both genomes, W represents the total weight difference between genes that match, and finally, N represents the number of connections / genes the genome with the larger number of connections has. For example, take the following 2 genomes:

[[1,.25][2,.55],[4,.78],[6,.2]]
and
[[1,.15][3,.92],[5,.37]]

Where the 0 index represents innovation number and the 1 index represents weight value. E  would be 1, since there is 1 excess gene, gene 6. D would be 4, since connections 2 and 4 are not in genome 2, and connections 3 and 5 are not in genome 1. W would be .10, since only connection 1 is shared between the two genomes.

The second formula is a bit more complicated. From my understanding, correct me if I'm wrong, this is a formula for adjusting fitness. f′i is the adjusted fitness, which will replace the original fitness, fi. For every genome j in the entire population, yes, entire population and not just every genome in its specie, it will calculate the distance between j and i, i being the genome of fitness fi. Then it will sum up all the distance    values, and divide the original fitness fi by the total distance sum, and set f'i to that. Next,

  Every species is
  assigned a potentially different number of offspring in proportion to the sum of adjusted fitnesses f`i of its member organisms.

This assigning of number of species offspring is used so that one specie can't take over the entire population, which is the whole point of speciation in the first place, so in conclusion, these two formulas are vital to the function and efficiency of the NEAT algorithm.


Searching keywords on web I have not studied machine learning or AI really, but my job sometimes requires me to automate stuff. Right now the requirement I have, seems to be under AI domain, but I am not sure about terminologies or how to go about it. I will really appreciate if someone can guide me about the direction I need to start from.
(PS: This question might not belong on this SE, in that case please direct me to suitable SE)

What I'm required to do is find references on web about a certain situation. As an example I'll use "Music", so I have to make a system which will search around the web (Google and Twitter mainly) to see if there is any news/mention/event related to Music that occurred today, if so how many references (i.e. how big of a deal it is making).
It is not the generic term music which is expected in the output, but the names of Musicians, i.e. in Music this and this Artist appeared this many times.
I have to give the number of references, and also provide the references in output so that one can read them in detail.

The challenges are

- One event can be covered by many websites, and there can be one main website that published the original story with full details, while others just spread the word around in summarized way.
How do you filter references to pick the most suitable one, to show in results to the system's user, because I can not give user ~50 references to manually read through, I have to give like 1-2 suitable reference
- I need to give the name of the artist. One site will have many words, how do I know which word is actually the artist's name? One option can be to have a pre compiled list of specific artists and just search for them individually. But this way, I can be missing new artists.

The challenges I have, must have been addressed by some existing algorithm or mechanism, I'll appreciate if someone can let me know what kind of algo etc I need to refer to or study to get the task done.

A parallel situation might be that of spam/not spam. The detection of spam by AI has been pretty successful, so there is an existing algorithm - classification. However while you have a possible approach you are still missing the key ingredient which is sufficient data to train the model on.

AI depends on a large amount of data to train the model. Ideally you will have a team of researchers available to read a (large?) number of sources and classify by hand whether the source is the original reference or just a repeater, and whether the topic is relevant. The more labelled and balanced data you have, the better the model and the better the results. Then, like the spam/not spam situation, you just apply your model and the best references pop out. You already have the key word of music, so you just use as candidates those sources that reference "music" and any other defining keywords.


Why does training an SVM take so long? How can I speed it up? I'm trying to create and test non-linear SVMs with various kernels (RBF, Sigmoid, Polynomial) in scikit-learn, to create a model which can classify anomalies and benign behaviors.
My dataset includes 692703 records and I use a 75/25% training/testing split. Also, I use various combinations of features whose dimensionality is between 1 and 14 features. However, the training processes of the various SVMs take much too long. Is this reasonable?
I have also examined the ensemble BaggingClassifier in combination with non-linear SVMs, by configuring the n_jobs parameter to -1; nevertheless, the training process proceeds again too slowly.
How can I speed up the training processes?

The most likely explanation is that you're using too many training examples for your SVM implementation.

SVMs are based around a kernel function. Most implementations explicitly store this as an NxN matrix of distances between the training points to avoid computing entries over and over again.

In your case, with 75% of 700,000 examples, this matrix will require approximately 250GB of RAM to store, which is more than you're likely to have in consumer hardware.

If your SVM implementation can avoid caching the values, you might get a speedup that way, or you might not (you'll waste a lot of time recomputing them).

A much better way to deal with this is to just not use all of the data, since most of it will be redundant from the SVM's perspective (it only benefits from having more data near the decision boundaries). A good starting place would be to randomly discard 90% of the training data, and see what performance looks like.


Relation between size of parameters and complexity of model with overfitting I'm reading the book Pattern Recognition and Machine Learning by Bishop, specifically the intro where he covers polynomial regression model. In short, let's say we generate $10$ data points using the function $\sin(2\pi x)$ and add some gaussian random noise to each observation. Now we pretend not knowing the generating function and try to fit a polynomial model to these points.

As we increase the degree of the polynomial, it goes from underfitting ($d=1,2$) to overfitting ($d=10$). One thing the author notes is that the higher the degree of the polynomial, the higher the values of the coefficients (parameters). This is my first doubt: why does the size of the coefficients increase with the polynomial degree? And why is the size of the parameters related to overfitting?

Secondly, he states that even for degree $10$, if we get sufficiently many data points (say $100$), then the high degree polynomial will no longer overfit the data and should have comparatively better generalization performance. Second doubt: Why is this so?

Regarding your first question, this is domain/task specific, and not always the case. My guess of why it happens in your situation (You did not specify your domain, so ill assume it's sometimes outside $(-1,1)$) is that higher order polynomials increase much faster than lower order ones so it may fall under the trap where the coefficients have to be large to handle it. (e.g. if $x=2.0$ then $ 2^5*x^2 = x^7 \rightarrow 32*x^2 = x^7$)

Now to answer why more points will cause less overfitting is sheerly because you have more equations. Your loss is generally some form of average distance accross points and predictions. Given $N$ points I can always find an $N-1$ order polynomial that fits it exactly. Instead of a formal proof I will provide a form in which you can easily generate this polynomial from $N$ $(x_i, y_i)$ pairs
$$ f(x) = \sum_{i=1}^{N}y_i\prod_{j=1\atop j \ != i}^N\frac{(x-x_j)}{(x_i-x_j)}$$

As you can see, for each $x_i$ it will zero out all terms in the sum except one where the product become 1 making $f(x_i) = y_i$ by construction. So adding more points will prevent whatever optimization process you are learning to directly solve each point exactly, but instead try to find an algorithm that best generalizes to each of the points.


How does an LSTM output the correct dimensions for classes? Take the below LSTM:

input: 5x1 matrix
hidden units: 256
output size (aka classes, 1 hot vector): 10x1 matrix

It is my understanding that an LSTM of this size will do the following:

$w_x$ = weight matrix at $x$

$b_x$ = bias matrix at $x$

activation_gate = tanh($w_1$ $\cdot$ input + $w_2$ $\cdot$ prev_output + $b_1$)

input_gate = sigmoid($w_3$ $\cdot$ input + $w_4$ $\cdot$ prev_output + $b_2$)

forget_gate = sigmoid($w_5$ $\cdot$ input + $w_6$ $\cdot$ prev_output + $b_3$)

output_gate = sigmoid($w_7$ $\cdot$ input + $w_8$ $\cdot$ prev_output + $b_4$)

The size of the output of each gate should be equal to the number of hidden units, ie, 256. The problem arrises when trying to convert to the correct final output size, of 10. If the forget gate outputs 256, then it is summed with the element wise product of the activation and input gate to find the new state, this will result in a hidden state of size 256. (Also in all my research I have not found anywhere whether this addition is actually addition, or simply appending the two matrices).

So if I have a hidden state of 256, and the output gate outputs 256, doing an element wise product of these two results in, surprise surprise, 256, not 10. If I instead ensure the output gate outputs a size of 10, this no longer works with the hidden state in an element wise product.

How is this handled? I can come up with many ways of doing it myself, but I want an identical replica of the basic LSTM unit, as I have some theories I want to test, and if it is even the slightest bit different it would make the research invalid.

After some more research, I have found the answer.

An LSTM is comprised of 1 LSTM cell that is continuously updated by passing in new inputs, the hidden state and the previous output. All nodes inside the LSTM cell are of size hidden_units. That means the output of the activation gate, input gate, forget gate and output gate are all of size hidden_units. As an extension of this, the size of the hidden state is also equal to hidden_units.

The problem arises when the desired output is not equal to that of hidden_units. This is fixed simple and easily, by slapping a basic feed forward net onto the end of the output of the LSTM, so even if it outputs 256, you can convert it to n nodes. These final output nodes are typically put through a softmax with cross-entropy loss applied.

I came to the conclusion by looking at the code included here: https://towardsdatascience.com/lstm-by-example-using-tensorflow-feb0c1968537

In which you can see the final output has its own separate weights and bias'.


How to train a LSTM model with multi dimensional data I am trying to train my model using LTSM layer in Keras (python). I have some problems regarding the data representation and feeding it into the model.

My data is 184 XY coodinates encoded as a numpy array with two dimensions: one corresponding to the X or Y and second is every single point of X or Y. Shape of a single spectrum is (2, 70). Altogether, my data has a dimension of (184, 2, 70).

The label set is an array of 8 elements which describes the percentage distribution of some 8 features which are describing XY. The shape of an output is (184, 8).

My question is how can I train using the time series for each XY pair and compare it to the corresponding label set? Different XY data show similar features to each other that is why it is important to use all 184 sample for the training. What would be the best approach to handle this problem? Below I show the schematics of my data and model:

Input: (184, 2, 70) (number of XY, X / Y, points)

Output: (184, 8) (number of XY, predictions)

I look forward for some ideas!

LSTM can be tricky, I'll give my $0.02.
LSTM input layer defines the shape so it would be something like this.
If I am understanding your question correctly, your data can be framed as 184 samples with 2 time steps and 70 features?
So the start of the code might look like this.
model = Sequential()
model.add(LSTM(184, input_shape=(50, 2)))


How can a system like Jarvis understand the commands and take actions? I am looking to make an AI like Jarvis. A perfect real-life example of this type of system is the simple AI that Mark Zuckerberg has recently built. Here is a description of how his AI works. From what I understand, the AI understands keywords, context, synonyms, and then from there decides what to do. Here is also a video of the AI in action.
I have many questions about how these systems works. Firstly, what necessary steps are required to gather the meaning of a input? Secondly, how does the system, once it extracts all of the necessary information on the input, determine what action it needs to take and what to say back to the user? Lastly, it also states that the system can learn the habits and preferences of the user. How can a system do this?

Jarvis was built using the suite of tools that facebook developers are constantly updating. The answer to this question is that there's no simple answer; it has a lot of moving parts.
Take for example natural language processing. There are a number of sub-topics that are each considered "big" problems, such as part-of-speech recognition, coreference resolution, sentiment analysis, relationship extraction, and many more. Tools have been built to tackle these various topics, but, to my knowledge, none of them really understand language, but rather statistically approximate it.
In the case of Jarvis, since it is a home-automation system, it's probably built with the user-given commands in mind from the beginning, so it's not trying to understand the whole human language, it's built to do some tricks.
Just keep in mind on your journey into this space that true AI like what we imagine will have some defining features like hierarchical representations of tasks and goal-orientation. If you really get into it I'd start with reinforcement learning, or try reading through the Society of Minds.


Perceptron learning algorithm: different accuracies for different training methods So, my question is a bit theoretical. I have been trying to implement a perceptron based classifier with outputs 1 and 0 depending on  the category. I have used 2 methods: The example by Example learning method and Batch learning method. I also have defined another method which will measure accuracy according to the formulae number_of_samples_classified_correctly/total_number_of_samples(I'm not sure this should be the correct definition for accuracy and you are welcome to suggest a better measure). Now there are a few confusions i'm facing. Firstly, the accuracy of example by example learning is different from batch learning by 2%. Also the best accuracy achieved in both cases is depending on the slopes. So where exactly is the mistake?(Batch learning algorithm=error*input_vector( where error can be 1,-1 or 0 ) summed over all input vectors and then added to weights).

For initial slope[1,-1] giving an accuracy of 88% example by example learning
For initial slope[1,-1] giving an accuracy of 88% batch learning
For initial slope[1,1] giving an accuracy of 84% example by example learning
For initial slope[1,1] giving an accuracy of 86% batch learning

In Brief:

re-train your dataset. I believe where you get lower accuracy scores, your model has not converged to the final state. duplicate your dataset multiple times and create a bigger one, then train your model with it.

In Detail:

  number_of_samples_classified_correctly/total_number_of_samples(I'm not sure this should be the correct definition for accuracy and you are welcome to suggest a better measure)

This is a valid accuracy metric. In fact if the value is acc, then 1-acc is called misclassification error. So your metric is good unless you have some class imbalance, where you need to use other metrics such as Cohen's Kappa score.

  the accuracy of example by example learning is different from batch learning by 2%. Also the best accuracy achieved in both cases is depending on the slopes.

I strongly believe both strange results happen because the number of your instances (i.e. examples) is low, or let's say your learning rate is small. As you know Perceptron algorithm corrects the weights of the decision hyper-plane by delta learning rule: it reads each instance, calculates the error (in case of binary classification {-1,0,1}) and updates the weights by c.x.E where:

c is learning constant/rate/step
x is the data instance
E is the error

Thus the weights (or slopes) change a little bit every time. There is no guarantee that your model reaches its final states after all the instances are given to it. Since when you start training with different initial weights, or change the order of instances given to your model, the distance or from or moving speed toward the final state changes. So as I mentioned above, re-train your model and let it converge to the final state OR increase your learning step (learning rate) . I believe all the 4 mentioned-above-accuracies will be the same then; I mean not only you should have the same model for different initial weights, but also for batch and online training.

Please update your post with new findings and question my claims if those were false.


What is the difference between hill-climbing and greedy best-first search algorithms? While watching MIT's lectures about search, 4. Search: Depth-First, Hill Climbing, Beam, the professor explains the hill-climbing search in a way that is similar to the best-first search. At around the 35 mins mark, the professor enqueues the paths in a way similar to greedy best-first search in which they are sorted, and the closer nodes expanded first.

However, I have read elsewhere that hill climbing is different from the best first search. What's the difference between the two then?

Let's see their definition first:

Best First Search (BFS):

  Best-first search is a search algorithm that explores a graph by expanding the most promising node chosen according to a specified rule.

  estimating the promise of node n by a "heuristic evaluation function ${\displaystyle f(n)}$ which, in general, may depend on the description of n, the description of the goal, the information gathered by the search up to that point, and most importantly, on any extra knowledge about the problem domain."

Hill Climbing (HC):

  In numerical analysis, hill climbing is a mathematical optimization technique that belongs to the family of local search. It is an iterative algorithm that starts with an arbitrary solution to a problem, then attempts to find a better solution by making an incremental change to the solution. If the change produces a better solution, another incremental change is made to the new solution, and so on until no further improvements can be found.

Base on the definition, we can find the following differences:

The aim of BFS is reaching to a specified goal by using a heuristic function (it might be greedy) vs. HC is a local search algorithm
BFS is mostly used in the graph search (in a wide state space) to find a path. vs. HC is using for the optimization task.


Given a query image Q and two other images X and Y, how to determine which one is most similar to Q? Given a query image Q and two other images X and Y (you can assume they have more or less the same resolutions if that simplifies the problem), which algorithm would perform extremely well at determining which image between X and Y is most similar to Q, even when the differences are rather subtle? For example, a trivial case would be:

Q = image of mountains, X = image of mountains, Y = image of dogs, therefore it is clear that sim(Q,X) > sim(Q,Y).

However, examples of trickier cases would be:

Q = image of a yellow car, X = image of a red car, Y = image of a yellow car, therefore sim(Q,Y) > sim(Q,X) (assuming the car shapes are more or less the same).
Q = image of a man standing up in the middle with a black background, X = image of another man standing up in the middle with a black background, Y = image of a woman standing up in the middle with a black background, therefore sim(Q,X) > sim(Q,Y).

Which algorithm (or combination of algorithms) would be robust enough to handle even the tricky cases with very high accuracy?

From your examples I assume you presuppose image recognition in the sense that you don't compare the actual images, but the descriptions of what the images contain.

For comparing images there are various algorithms working on the visual similarity. This can sometimes lead to interesting results, as you probably have seen images on the internet like "dog or muffin". A purely visual approach would find this hard to do.

However, if you do have the description of the image (as there are ways of getting captions from images), then it would just be a text comparison between three sentences: the one describing your query image, and those describing your images X and Y. There are ways of getting at the semantic similarity of sentences. The simplest way (from your examples) would be to look at the overlap in words: [yellow, car]/Q, [red, car]/X, and [yellow, car]/Y obviously has the largest overlap with Q and Y. This is rather simplistic, but it is what your examples suggest you are dealing with.

But, getting a proper description from an image is a hard task, of the calibre that Google and Instagram are still working on; they have large amounts of training data and huge resources they can throw at the problem. So, unless that is your starting point, it will not be easy to achieve.

Overall I do not think it is currently possible to solve this problem with high accuracy.


Could curiosity improve artificial intelligence? While thinking about AI, this question came into my mind. Could curiosity help in developing a true AI? According to this website (for testing creativity):

Curiosity in this context refers to persistent desire to learn and discover new things and ideas. A curious person

always looks for new and original ways of thinking,
likes to learn,
searches for alternative solutions even when traditional solutions are present and available,
enjoys reading books and watching documentaries,
wants to know how things work inside out.

Let's take Clarifai, an image/video classification startup, which can classify images and video with the best accuracy (according to them). If I understand correctly, they trained their deep learning system using millions of images with supervised learning. In the same algorithm, what would happen if we somehow added a "curiosity factor" when the AI has difficulty in classifying an image or its objects? It would ask a human for help, just like a curious child.
Curiosity makes a human being learn new things and also helps to generate new original ideas. Could the addition of curiosity change Clarifai into a true AI?

Does this addition of curosity changes clarifai into a true AI?

As per my answer to this question, we don't know what the ingredients for a 'true AI' are. Via the Turing Test and its variants, the best we can do is "know one when we see one".

Curiosity certainly appears necessary for intelligence, though it doesn't seem sufficient - a lemming-like creature curious to see what's at the bottom of a steep cliff might not survive long enough to learn caution, even if it had the learning mechanisms to do so.

Here is some work by Schmidhuber on Artificial Curiousity.

Pierre-Yves Oudeyer has also done quite a lot of work on this and Active Learning/Intrinsic motivation.


What is sample efficiency, and how can importance sampling be used to achieve it? For instance, the title of this paper reads: "Sample Efficient Actor-Critic with Experience Replay".

What is sample efficiency, and how can importance sampling be used to achieve it?

An algorithm is sample efficient if it can get the most out of every sample. Imagine learning trying to learn how to play PONG for the first time. As a human, it would take you within seconds to learn how to play the game based on very few samples. This makes you very "sample efficient". Modern RL algorithms would have to see $100$ thousand times more data than you so they are, relatively, sample inefficient.

In the case of off-policy learning, not all samples are useful in that they are not part of the distribution that we are interested in. Importance sampling is a technique to filter these samples. Its original use was to understand one distribution while only being able to take samples from a different but related distribution. In RL, this often comes up when trying to learn off-policy. Namely, that your samples are produced by some behaviour policy but you want to learn a target policy. Thus one needs to measure how important/similar the samples generated are to samples that the target policy may have made. Thus, one is sampling from a weighted distribution which favours these "important" samples. There are many methods, however, for characterizing what is important, and their effectiveness may differ depending on the application.

The most common approach to this off-policy style of importance sampling is finding a ratio of how likely a sample is to be generated by the target policy. The paper On a Connection between Importance Sampling and
the Likelihood Ratio Policy Gradient (2010) by Tang and Abbeel covers this topic.


Can we make Object Detection as human eyes+brain do? I am so much curious about how do we see(with eyes ofc) and detect things and their location so quick. Is the reason that we have huge gigantic network in our brain and we are trained since birth to till now and still training.
basically I am saying , are we trained on more data and huge network? is that the reason?
or what if there's a pattern for about how do we see and detect object.
please help me out, maybe my thinking is in wrong direction.
what I wanna achieve is an AI to detect object in picture in human ways.thanks.

Object detection can conceivably imitate what the human visual system does.  Research along these lines began in the 1980s in multiple laboratories and was termed Computer Vision.

  How do we see ... and detect things and their location so quick[?]  Is the reason that we have huge gigantic network in our brain and we are trained since birth to till now and still training[?]

Neurological visual systems continue to train as long as the eyes and neurons continue to function.  A person may, long after doing well in sports, learn how to catch glass objects that fall off a counter using peripheral vision, using the same neural pathways as were used for visually coordinated movement in sports but.  They are newly trained to the new cognitive intention of saving glassware and clean-up time.

The mapping of signals from the rods and cones to the visual cortex has been studied and some generalities have influenced artificial vision research and development.  The convolution kernels used in CNNs are partly derived from biological research.  In summary, the way mammals and other organisms with the equivalents of retinas see is via a set of transformations and pattern matching circuits.  The sequence of information types through biological visual networks can be summarized.

Light as a function of compound angle and time.
Edges in both space and time
Elements of objects and actions — These are real world features, not in the machine learning sense but in the natural language use of the term features.
Objects and actions

The development of object and action recognition in computers follows this same basic sequence.  The goal stated in the question is this.

  Achieve an AI to detect object in picture in human ways

Something like this has been the goal of many government, academic, and corporate laboratories, however the ability to recognize objects in still pictures is not as valuable as detecting the trajectories and other expressions of objects (such as changes in facial expressions) in real time.

Automated aiming, piloting, walking, and driving have been computer vision objectives since the first control systems were developed for anti-aircraft defense systems.  Considerable effort is being invested into how steering, breaking, and signaling can be automated in road vehicles.  A large array of household and industrial devices are being developed that rely on mapping the environment and navigating a robotic device through the map.

Stationary object recognition work is largely surrounding either categorizing images or drawing vector graphics from raster images, but these functions have limited application.  As mobile devices develop further and images on the web give way to videos on the web, a very strong and uninterrupted trend, the temporal dimension will continue to gain importance in the field computer vision.

None of these objectives are novel.  They are all based on animal capabilities, only some of which are uniquely human, and have been the subject of research since before the advent of digital computers.  However the feasibility of many designs has improved, largely due to a significant reduction in costs for fixed computing resources.  The vision system hardware of today costs roughly one thousandth of its 1990 cost.


How to label training data for YOLO I am having a question on how to label training data for YOLO algorithm.
Let's say that each label Y, we need to specify $[P_c, b_x, b_y, b_h, b_w]$, where $P_c$ is the indicator for presence (1=present, 0=not present), $(b_x, b_y)$ is the relative position of the center of the object-of-interest, and $(b_h, b_w)$ is the relative dimension of the bounding box containing the object.
Using picture below as an example, the cell (1,2), which contains a black car, should have a label $Y = [1, 0.4, 0.3, 0.9, 0.5]$. And for any cells without cars, they should have a label $[0, ?, ?, ?, ?]$ [Coursera Deep Learning Specialization Materials]

But if we have a finer grid like this, where the dimension of each cells is smaller than the ground truth bounding box.

Let's say that the ground truth bounding box for the car is the red box, and the ground truth center point is the red dot, which is in cell 2.
For cell 2 it will have label $Y = [1, 0.9, 0.1, 2, 2]$, is this correct? And for cell $1, 3, 4$ what kind of label will they have? Do they have $P_c=1$ or $P_c = 0$? And if $P_c=1$, how will the $b_x$ and $b_y$ be? (As I remember that $b_x, b_y$ should have value between $0$ and $1$. But in cell $1,3,4$ there is no center point of the object-of-interest)

In effect, the midpoint is contained in cell $2$. Cells $1,3,4$ will be shown at $P_c=0$ according to the YOLO algorithm, which only takes in count the cell that contains the midpoint and calculates the bounding box, as you mentioned, with $b_y b_x, b_y, b_h, b_w$. With the proposition of $Y = [1, 0.9, 0.1, 2, 2]$, I would think that if you take the point $(0,0)$ as the left-upper corner, $Y$ would be more likely to be $Y = [1,0.2,0.57,0.21,0.15]$, being calculated with a grid of $19 \times 19$.


How can I go from $R(s)$ to $R(s,a)$ in this specific MDP? I'm trying to implement a research paper, as explained in this other post, here the author of the paper assumed R as a function of both states and actions, while the code (and the MDP) I'm using to test this algorithm assumes R as a function of only states.
My question is:
Given $\mathcal{X}$ as the set of states of an MDP and $\mathcal{A}$ as the set of actions of an MDP. Supposing I have four states ($1$,$2$,$3$,$4$), two actions $a$ and $b$ and a reward function $R: \mathcal{X}\to\mathbb{R}$ s.t.
$R(1) = 0$
$R(2) = 0$
$R(3) = 0$
$R(4) = 1$
If I need to change the current reward function to a new reward function $R:\mathcal{X}\ \times \mathcal{A} \to\mathbb{R}$ is it ok to compute it as $\forall a,R(s,a) = R(s)$?
$R(1,a) = 0$
$R(1,b) = 0$
$R(2,a) = 0$
$R(2,b) = 0$
$R(3,a) = 0$
$R(3,b) = 0$
$R(4,a) = 1$
$R(4,b) = 1$
More generally, what's the correct way of generalising a reward function
$R: \mathcal{X}\to\mathbb{R}$ to a reward function $R:\mathcal{X}\ \times \mathcal{A} \to\mathbb{R}$?

As explained here, I can write $R(s,a) = R(s)\  \forall a$ since the reward of my specific MDP is dependent exclusively to the state $s$.


Why doesn't VAE suffer mode collapse? Mode collapse is a common problem faced by GANs. I am curious why doesn't VAE suffer mode collapse?

With Generative Adversarial Networks, all the generator cares about is fooling the discriminator. There's no requirement to be clever, or exhaustive, or make efficient use of the input space. As long as the discriminator returns "real" (vs. "fake") the generator "wins".

The hope is that as the generator and discriminator are trained simultaneously, each will attempt to capture the faults of the other. As the discriminator gets better for one mode in telling real from fake, the generator will focus more on the other modes where it will have an easier time fooling the discriminator on.

But that isn't the only way things can happen. When the discriminator gets better on distinguishing one mode, the generator can instead concentrate on that mode. Instead of spreading its training over different modes, it can instead attempt to get really good at producing samples within a particular mode. As long as the discriminator can be fooled into reporting "real" for the samples being produced, the generator gets the same reward as if it were to spread itself across multiple modes.

Think about the forger/detective analogy. Instead of a forger trying to learn how to be good at making a range of paintings (Caravaggios, van Goghs, Warhols, etc.) the forger can concentrate on being really good at making a particular style of painting (e.g. Dutch Masters) or even a particular artist. Someone who spends all their time perfecting their Rembrandt forgeries will do a much better job evading detection than someone who tries to be good at both Picasso and Rembrandt (and Klee and Miró and ...)

Variational Autoencoders are different, though. With a VAE, the latent space is built from all of the input examples. There's no way for a VAE to concentrate on a particular mode. All of the training examples are being mapped to the same latent space. If, for some reason, the decoder network decided to concentrate on a single mode, mapping the latent space to outputs of just that mode, then all the training examples from the other modes would have poor recovery. This means there would be a strong loss and gradient correcting that for the decoder training. The decoder training is balanced across all the input examples/modes, so the decoder needs to be similarly balanced when decoding the latent space. GANs don't have this similar balance, as the generator doesn't necessarily need to be balanced in its output - just so long as it can reliably fool the discriminator.

That doesn't mean that VAEs don't have similar issues, though they're not identical to mode collapse. The problem with VAEs is that each individual example is (in extremis) being mapped to the same random distribution in the latent space. This can encourage the decoder to ignore the latent variable input, generating output more-or-less arbitrarily. There's a number of approaches recently (in particular Adversarial Autoencoders as well as InfoVAE and the closely related Wasserstein Autoencoders) which attempt to remedy this by constraining the latent space distribution on an ensemble level, rather than on an individual training example basis.


Simulating successful trajectories in Montezuma's Revenge turns out to be unsuccessful I have written code in OpenAI's gym to simulate a random playing in Montezuma's Revenge where the agent randomly samples actions from the action space and tries to play the game. A success for me is defined as the case when the agent is atleast able to successfully retrieve the key (Gets a reward of 100). And such cases I dump in a pickle file. I got 44 successful cases when I kept it to run for a day or so. Here is the code I use to generate the training set :

import numpy
import gym
import cma
import random
import time
import pickle as pkl

env   = gym.make('MontezumaRevenge-ram-v0')

observation = env.reset()

#print(observation)
#print(env.action_space.sample())

obs_dict    = []
action_dict = []
success_ctr = 0

for i in range(0, 1000000):
    print('Reward for episode',i+1)
    done = False
    rew = 0
    action_list = []
    obs_list    = []
    while not done:
        action = env.action_space.sample()
        observation, reward, done, _ = env.step(action)
        action_list.append(action)
        obs_list.append(observation)

        rew += reward
        env.render()
        time.sleep(0.01)
        if done:
            env.reset()
            if rew > 0:
                success_ctr += 1
                print(action_list)
                action_dict.append(action_list)
                obs_dict.append(obs_list)
                pkl.dump(obs_dict, open("obslist.pkl", "wb"))
                pkl.dump(action_dict, open("action.pkl", "wb"))

    print(rew)
    time.sleep(1)

try:
    print(obs_dict.shape)
except:
    pass

print("Took key:", success_ctr)

I loaded the successful cases from my generated pickle file, and simulated the agent's playing using those exact same cases. But, the agent never receives a reward of 100. I dont understand why this is happening. A little search online suggested it could be because of noise in the game. So, I gave a sleep time, before running each episode. Still, doesn't work. Can someone please explain why is this happening? And suggest a way I could go about generating the training set?

What's exactly the point of time.sleep() in this code? I don't really understand it, you're simply stopping the execution of the program for $0.01$ seconds, how will that affect the simulator in any way ? It's not running in parallel, it does one step of the simulation when you call env.step function and returns the next state and reward. Calling sleep function only slows down the program here.

The reason for the failure of successful trajectories, when repeated, is probably because the environment isn't stationary. That means that there are enemies or obstacles moving in the environment. If you simply repeat the trajectory that was successful once the enemies and obstacles might be in different positions and the agent will die. The reason why it succeeded the first time is because the agent got lucky. These are still valid learning trajectories because they were successful. The agent should learn reasons (features) why those trajectories were successful but not learn the trajectories themselves because, as you saw, they don't generalize well. If you plan on doing some kind of supervised learning approach you should also generate variety of unsuccessful trajectories so that the agent can learn what are correct and what are not correct actions depending on the current state in the environment.


It is mathematically correct to use a Policy Gradient method for 1-step trajectories? I have come across a Google paper that uses the REINFORCE algorithm (a Policy Gradient Method) for a case where the trajectory of the episodes it proposes would be only one step.
When trying to replicate the experiments they propose I found that there are some problems with the stability of the method (maybe that's why it is not accepted by peer review).
Researching on my own I have found something I suspected, and that is that the problem they present could be solved as a multiarmed bandit problem in THIS link. But because of this event, I have the doubt if using methods based on trajectories (such as Policy Gradient Methods) has some mathematical problem in situations where the trajectory is a single step.
PS: I think the problem of this paper may be also that they average only after one execution of a trajectory and not over k trajectories as it is necessary for a Policy Gradient Method, so I would also like to know the opinion of more people about this issue.

The fundamental idea behind policy gradient is just to maximise the return averaged across all probably trajectories, i.e
$$\begin{align} J(\theta) &= E[\sum\limits_{t=1}^{\tau}r(s_t,a_t)]\\
&=E_{\tau\sim p(\tau)}[R(\tau)]
\end{align}$$
Where $\tau$ represents the probability of selecting a particular trajectory, if the trajectories all have fixed length then $\tau$ only has non-zero probability for trajectories of the specified length which in this case is 1.
The REINFORCE algorithm takes this expression and with some simplifications (causality to improve variance) and manipulation (log trick) obtains the gradient, pretty much as simple as that.

Intuition of algorithm in paper
In the algorithm they denote the $\pi_\theta$ (which is typically reserved for describing the policy) as the probability of the selection vector. By considering instead that the function $h_{\theta}$ as the policy instead I think it can be seen that they are actually averaging over multiple trajectories.
So we instead think of the data points as state-action pairs that we pass into $h_\theta$ to get the probability of selecting said action for a given state. These probabilities then dictates whether we choose the action or not. An alternative way to interpret this as if we imagine the action space for each state as binary then we can think of the "other action" as not impacting the predictor.
The gradient used for updating the parameters associated with the policy uses the log probability of $\pi_\theta$ which if we expand it, expressing it using $h_\theta$, (as done on page 4)  we can see it's the sum of the log probabilities of selecting (or not selecting) each data point.
By considering the return for each data point constant (each data point has the same loss incurred by predictor model on the validation set) and absorbing the average over batch size into the step size $\beta$ it could be interpreted as an average

Stability issues
RL is plagued with stability issues, be it selection of hyper parameters, random seeding etc. It's hard to often pinpoint why results aren't exact but as long as you get something in a similar ball park i'd say thats pretty good going


Could there be existential threats to humanity due to AI? We are doing research, spending hours figuring out how we can make real AI software (intelligent agents) to work better. We are also trying to implement some applications e.g. in business, health and education, using the AI technology.

Nonetheless, so far, most of us have ignored the "dark" side of artificial intelligence. For instance, an "unethical" person could buy thousands of cheap drones, arm them with guns, and send them out firing on the public. This would be an "unethical" application of AI.

Could there be (in the future) existential threats to humanity due to AI?

There is no doubt that AI has the potential to pose an existential threat to humanity.

  The greatest threat to mankind lies with superintelligent AI.

An artificial intelligence that surpasses human intelligence will be capable of exponentially increasing its own intelligence, resulting in an AI system that, to humans, will be completely unstoppable.

At this stage, if the artificial intelligence system decides that humanity is no longer useful, it could wipe us from the face of the earth.

As Eliezer Yudkowsky puts it in Artificial Intelligence as a Positive and Negative Factor in Global Risk,

"The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else."

  A different threat lies with the instruction of highly intelligent AI

Here it is useful to consider the paper clip maximiser thought experiment.

A highly intelligent AI instructed to maximise paper clip production might take the following steps to achieve its goal.

1) Achieve an intelligence explosion to make itself superintelligent (this will increase paperclip optimisation efficiency)

2) Wipe out mankind so that it cannot be disabled (that would minimise production and be inefficient)

3) Use Earth's resources (including the planet itself) to build self replicating robots hosting the AI

4) Exponentially spread out across the universe, harvesting planets and stars alike, turning them into materials to build paper clip factories

  Clearly this is not what the human who's business paperclip production it was wanted, however it is the best way to fulfil the AI's instructions.

This illustrates how superintelligent and highly intelligent AI systems can be the greatest existential risk mankind may ever face.

Murray Shanahan, in The Technological Singularity, even proposed that AI may be the solution to the Fermi paradox: the reason why we see no intelligent life in the universe may be that once a civilisation becomes advanced enough, it will develop an AI that ultimately destroys it.
This is known as the idea of a cosmic filter.

In conclusion, the very intelligence that makes AI so useful also makes it extremely dangerous.

Influential figures like Elon Musk and Stephen Hawking have expressed concerns that superintelligent AI is the greatest threat we will ever have to face.

Hope that answers your question :)


Why does the transformer do better than RNN and LSTM in long-range context dependencies? I am reading the article How Transformers Work where the author writes

  Another problem with RNNs, and LSTMs, is that it’s hard to parallelize the work for processing sentences, since you have to process word by word. Not only that but there is no model of long and short-range dependencies.

Why exactly does the transformer do better than RNN and LSTM in long-range context dependencies?

I'll list some bullet points of the main innovations introduced by transformers , followed by bullet points of the main characteristics of the other architectures you mentioned, so we can then compared them.
Transformers
Transformes (Attention is all you need) were introduced in the context of machine translation with the purpose to avoid recursion in order to allow parallel computation (to reduce training time) and also to reduce drop in performances due to long dependencies. The main characteristics are:

Non sequential: sentences are processed as a whole rather than word by word.
Self Attention: this is the newly introduced 'unit' used to compute similarity scores between words in a sentence.
Positional embeddings: another innovation introduced to replace recurrence. The idea is to use fixed or learned weights which encode information related to a specific position of a token in a sentence.

The first point is the main reason why transformer do not suffer from long dependency issues. Original transformers do not relies on past hidden states to capture dependencies with previous words, they process a sentence as a whole, reason why there is no risk to loose (or 'forget') past information. Moreover, also multi-head attention and positional embeddings both provide information about the relationship between different words.
RNN / LSTM
Recurrent neural networks and Long-short term memory models for what concern this question are almost identical in their core properties:

Sequential processing: sentences must be processed words by words.
Past information retained through past hidden states: sequence to sequence models follow the Markov property, each state is assumed to be dependent only on the previously seen state.

The first property is the reason why RNN and LSTM can't be trained in parallel. In order to encode the second word in a sentence I need the previously computed hidden states of the first word, therefore I need to compute that first. The second property is a bit more subdole, but not hard to grasp conceptually. Information in RNN and LSTM are retained thank's to previously computed hidden states. The point is that the encoding of a specific word is retained only for the next time step, which means that the encoding of a word strongly affect only the representation of the next word, its influence is quickly lost after few time steps. LSTM (and also GruRNN) can boost a bit the dependency range they can learn thanks to a deeper processing of the hidden states through specific units (which comes with an increased number of parameters to train) but nevertheless the problem is inherently related to recursion. Another way in which people mitigated this problem is to use Bi-directional models, which encode the same sentence from two direction, from the start to end and from the end to the start, allowing this way words at the end of a sentence to have stronger influence in the creation of the hidden representation, but this is just a workaround rather than a real solution for very long dependencies.
CNN
Also convolutional neural network are widely used in nlp since they are quite fast to train and effective with short texts. The way they tackle dependencies is by applying different kernels to the same sentence, and indeed since they're first application to text (Convolutional Neural Networks for Sentence Classification) they were implement as multichannel CNN. Why do different kernels allow to learn dependencies? Because a kernel of size 2 for example would learn relationships between pairs of words, a kernel of size 3 would capture relationships between triplets of words and so on. The evident problem here is that the number of different kernels required to capture dependencies among all possible combination of words in a sentences would be enormous and unpractical because of the exponential growing number of combinations when increasing the maximum length size of input sentences.
To summarise, Transformers are better than all the other architectures because they totally avoid recursion, by processing sentences as a whole and by learning relationships between words thank's to multi-head attention mechanisms and positional embeddings. Nevertheless, it must be pointed out that also transformers can capture only dependencies within the fixed input size used to train them, i.e. if I use as a maximum sentence size 50, the model will not be able to capture dependencies between the first word of a sentence and words that occur more than 50 words later, like in another paragraph. New transformers like Transformer-XL tries to overcome exactly this issue, by kinda re-introducing recursion by storing hidden states of already encoded sentences to leverage them in the subsequent encoding of the next sentences.


How to find the category of a technical text on a surface-semantic-level There are some predefined categories( Overview, Data Architecture, Technical Details, Applications, etc). The requirement is to classify the input text of paragraphs into their resp. category. I can't use any pre-trained word embeddings (Word2Vec, Glove) because the data entered is not in general English ( talking about dogs, environment, etc) but pure technical (How does a particular program orks, steps to download anaconda, etc). Don't have any data available on the internet to train as well. Anything that understands semantic-surface-level of a sentence will work

Problem Statement

Find the category of technical text on a surface-semantic-level.
The requirement is to classify the input text of paragraphs into their respective categories.

The categories given as predefined are as follows.

Overview
Data Architecture
Technical Details
Applications
etc

Some document types that would be added in place of 'etc' might include these.

Requirements
High Level Design
System Architecture
Testing Plan
Deployment Plan
Disaster Recovery Plan

Semantic Structure of Technical Documents
It is correct that the terminology between a requirements document and data architecture will likely have much in common.  More precisely, the distributions of linguistic elements of all documents common to a given system or project are likely to contain domain commonalities.  A system that pilots a drone will have the linguistic elements "drone(s)", "hover(ing|s)", "target(ing|s)", "flight plan(s)" in similar distributions throughout its documentations.
Indicators
It is likely that these five distinguishing characteristics can be exploited in categorization.

Sentence semantics
Inserted diagrammatic and pictorial conventions
Header conventions
Linguistic elements that appear commonly in only one type of technical document
Elements and structure from copying previous documents and modifying them or the use of boiler plates and templates in specific departments

Rather than focus entirely on text recognition and abandoning the other items in the list above would be unwise.  Diagrams, such as network diagrams and UML diagrams, may be quite easy to discern using deep convolutional approaches and would clearly identify the category to which the hosting document belongs.  That is also true of test case tables.
Recognizing that the proportions between the five indicators above and sections of indicators are variables in the model upon which training is applied will produce the best results.  For instance, the final paragraph may be more telling than the rest of the body.
Also, be aware that one can couple the proportional appearance of language elements within all text documents on record with the language elements identified in the example and then in later use of the trained model.  The training is likely to progress faster and produce more accurate and reliable results if features include an indication that the linguistic element "test case(s)" appearing in the input text is significantly less prevalent in the domain of technical documentation than the linguistic element "-ing" to indicate continuous tenses of verbs.
Avoid Static Grammars
Language parsers based on fixed language rules (grammars) have not had much success compared to association based semantic mapping, and linguistics has moved away from those static models for similar reasons.  Avoid grammar based parsing.
Existing Work
For the textual categorization, the below academic publications are some of the recent work that have already gained some notoriety.

Semantic clustering and convolutional neural network for short text categorization, P Wang, J Xu, B Xu, C Liu, H Zhang, F Wang… - Proceedings of the 53rd, 2015
Document Modeling with Gated Recurrent Neural Network for Sentiment Classification, D Tang, B Qin, T Liu - Proceedings of the 2015 conference on empirical, 2015
Learning Semantic Representations of Users and Products for Document Level Sentiment Classification*, D Tang, B Qin, T Liu - Proceedings of the 53rd Annual Meeting of the, 2015
Effective use of word order for text categorization with convolutional neural networks, R Johnson, T Zhang - arXiv preprint arXiv:1412.1058, 2014 - arxiv.org
Jumping NLP curves: A review of natural language processing research, E Cambria, B White - IEEE Computational intelligence, 2014
Recurrent Convolutional Neural Networks for Text Classification, S Lai, L Xu, K Liu, J Zhao - AAAI, 2015

Faulty Approaches
The approach in the comment of comparing the semantics of sentences will only determine one aspect of information redundancy between two sentences.
The number of comparisons are also a consideration.  For $\chi$ documents each containing $\sigma$ sentences would require $\chi \, (\chi - 1) \, \sigma \, (\sigma - 1)$ comparisons.  For $\chi = 10,000$ documents containing $\sigma = 1,000$ sentences, we have 99,890,010,000,000 sentence comparisons, the totality of which provides no particularly useful information about the category of any of the documents.
The documents must be related to a concept class, not each other.
Visualizations of the semantics aren't particularly useful unless you are looking for something and the visualization is designed to present that which is sought.
A Better Plan

Determine the number of example training documents, $t$, needed to be categorized by experts to create a sufficiently large training data set. (Consider using the PAC learning framework designed for this purpose.)
Draw that number from the full set of documents, using an appropriate random or highly pseudo-random method, herein referred to as method $\mathbb{D}$. That is the training example set.
Draw that number from the full set of documents again, using method $\mathbb{D}$. That is the test example set.
Have experts label (categorize) both example sets according to technical document type.  Use the same experts for both training and testing, and have them alternate periodically between the two sets so that their learning, fatigue, or boredom curves have a minimal effect on their categorizing of the two sets.
Profile each of the category-labeled documents in terms of the above five indicators.  Note that to use the images embedded in the document as part of the profiling, which could drastically improve system reliability, the images must be run through a separate diagram categorizing network trained separately on images to find diagrams characteristic of one of the technical document types.  That may sound like much work, but consider that categorizing drawing types is a well developed science and the labeling is only of the diagram types that are representative of particular document types.
Train an appropriately designed artificial network to categorize the documents, using the profiling results as example inputs and the labels from the categorizing from the experts.
Test using the test set
Based on the results of the test, decide whether to use the current training or re-execute a previous step using information gained from the first training and train again
Run the trained network on the full document set
Pull a sample from the result using method $\mathbb{D}$ to validate effective completion of the run


How many layers exists in my neural network? I have a neural network model defined as below. How many layers exist there? Not sure which ones to count when we are asked about the number.

def create_model():
    channels = 3
    model = Sequential()
    model.add(Conv2D(32, kernel_size = (5, 5), activation='relu', input_shape=(IMAGE_SIZE, IMAGE_SIZE, channels)))
    model.add(MaxPooling2D(pool_size=(2,2)))
    model.add(BatchNormalization())
    model.add(Conv2D(64, kernel_size=(3,3), activation='relu'))
    model.add(MaxPooling2D(pool_size=(2,2)))
    model.add(BatchNormalization())
    model.add(Conv2D(128, kernel_size=(3,3), activation='relu'))
    model.add(MaxPooling2D(pool_size=(2,2)))
    model.add(BatchNormalization())
    model.add(Conv2D(256, kernel_size=(3,3), activation='relu'))
    model.add(MaxPooling2D(pool_size=(2,2)))
    model.add(BatchNormalization())

    model.add(Conv2D(64, kernel_size=(3,3), activation='relu'))
    model.add(MaxPooling2D(pool_size=(2,2)))
    model.add(BatchNormalization())
    model.add(Dropout(0.2))
    model.add(Flatten())
    model.add(Dense(256, activation='relu'))
    model.add(Dropout(0.2))
    model.add(Dense(128, activation='relu'))
    model.add(Dense(2, activation = 'softmax'))

    return model

tl;dr I'd say your model has 8 layers (5 conv, 3 dense), however a lot of people count layers in other ways. From what I've seen this is by far the most conventional way for counting layers.

Justification

This is an interesting question because its quite subjective. In most cases only the convolutional and dense layers would count from your network. Bach norm, dropout and flatten are usually considered as operations to other layers rather than layers of their own (much like activation functions).

Note: It is debatable whether or not pooling layers are considered to be layers (as they have no trainable parameters) but in most cases they are not considered to be so.

Note 2: Batch norm, on the other hand, isn't usually considered a layer even though it has trainable parameters. Clearly the authors didn't introduce it as a layer, but as a way to normalize, shift and scale the inputs of a layer. This is apparent in some of the examples below which don't count batch norm as an actual layer.

Note 3: Conventionally all networks are considered to have [at least] one input layer but it doesn't count as a layer.

Examples

Some examples that follow this reasoning when counting layers are the following. I'll also write the pooling layers in each, but they clearly aren't considered as layers be the authors. When available I'll also write the number of layers that keras registers from their official implementations:

The ResNet-50 architecture has 50 layers (49 conv, 2 pool, 1 dense), however keras registers it as 177 layers. ResNets also use batch normalization after each convolution (so 49 batch norms in total), but clearly don't count them as layers.

The Resnet-34 has 34 layers (33 conv, 2 pool, 1 dense). Like the previous, this also uses batch norm but doesn't count it.

VGG-19 has 19 layers (16 conv, 5 pool 3 dense). Keras registers this as 26 layers.

AlexNet is considered to have 8 layers (5 conv, 3 pool, 3 dense).