What makes the animal brain so special? Whenever I read any book about neural networks or machine learning, their introductory chapter says that we haven't been able to replicate the brain's power due to its massive parallelism.
Now, in modern times, transistors have been reduced to the size of nanometers, much smaller than the nerve cell. Also, we can easily build very large supercomputers.

Computers have much larger memories than brains.
Computes can communicate faster than brains (clock pulse in nanoseconds).
Computers can be of arbitrarily large size.

So, my question is: why cannot we replicate the brain's parallelism if not its information processing ability (since the brain is still not well understood) even with such advanced technology? What exactly is the obstacle we are facing?

One probable hardware limiting factor is internal bandwidth.  A human brain has $10^{15}$ synapses.  Even if each is only exchanging a few bits of information per second, that's on the order of $10^{15}$ bytes/sec internal bandwidth.  A fast GPU (like those used to train neural networks) might approach $10^{11}$ bytes/sec of internal bandwidth.  You could have 10,000 of these together to get something close to the total internal bandwidth of the human brain, but the interconnects between the nodes would be relatively slow, and would bottleneck the flow of information between different parts of the "brain."
Another limitation might be raw processing power.  A modern GPU has maybe 5,000 math units.  Each unit has a cycle time of ~1 ns, and might require ~1000 cycles to do the equivalent processing work one neuron does in ~1/10 second (this value is totally pulled from the air; we don't really know the most efficient way to match brain processing in silicon).  So, a single GPU might be able to match $5 \times 10^8$ neurons in real-time.  You would optimally need 200 of them to match the processing power of the brain.
This back-of-the-envelope calculation shows that internal bandwidth is probably a more severe constraint.


Is there a system that automatically designs a set of rules from the examples? Being by no means an expert in the field, I do however know of solvers such as Z3 that can generate a correct program, provided you can express your constraints as a set of logical rules, such as here.
However, in the example given above, logical rules are designed "by hand" from a set of examples, so the "compilation step" from the examples to the set of rules must be done in the user's head.
Does a system automating this "compilation step" exist (i.e automated rule set deduction from a set of examples), that would allow the user to see and make use of the set of rules?

This is an active research topic.
Consider reading

J.Pitrat's Artificial Beings, the conscience of a conscious machine book

This Understanding machine learning book

old Lenat's papers on Eurisko.

and recent papers such as Learning Structured Embeddings of Knowledge Bases and several others in Artificial Intelligence Journal

about the MILEPOST GCC and Ctuning projects

this paper on Reconciling deep learning with symbolic artificial intelligence: representing objects and relations

My hope is to get funded so that RefPerSys could (in a few years) be able to deduce some rules and metarules.


Why the cost/loss starts to increase for some iterations during the training phase? I am trying to build a recurrent neural network from scratch. It's a very simple model. I am trying to train it to predict two words (dogs and gods). While training, the value of cost function starts to increase for some time, after that, the cost starts to decrease again, as can be seen in the figure.

I am using the gradient descend method for optimization. Decreasing the step size/learning rate does not change the behavior. I have checked the code and math, again and again, I don't think there is an error (I could be wrong).
Why is the cost function not decreasing monotonically? Could there be a reason other than an error in my code/math? If there is an error, do you think that it is just a coincidence that each time the system finally converges to a very small value of error?
I am a beginner in the field of machine learning, hence, many questions I have asked may seem foolish to you. I am saving the values after every 100 iterations so the figure is actually for 15000 iterations.
About training:
I am using one-hot encoding. As the training data has only two samples ("gods" and "dogs"), where each alphabet is represented as d=[1,0,0,0],o=[0,1,0,0],g=[0,0,1,0],s=[0,0,0,1]. The recurrent neural network (RNN) goes back to a maximum of 3 time units, (e.g for dogs, the first input is 'd', then 'o', followed by 'g' and s). So, for the second input, the RNN goes back to 1 input, for the third input the RNN observes both previous inputs and so on. After calculating the gradients for the word "dogs", the values of the gradients are saved and the process is repeated for the word "gods". The gradients calculated for the second input "gods" are summed with the gradients calculated for "dogs" at the end of each epoch/iteration, and then the sum is used to updated all the weights. In each epoch, the inputs remain the same i.e "gods" and "dogs". In mini-batch training, in some epoch the RNN may encounter new inputs, hence, the loss may increase. However, I do not think that what I am doing qualifies as mini-batch training as there are only two inputs, both inputs are used in each epoch, and the sum of calculated gradients is used to update the weights.

In general, there's nothing wrong with training loss to increase from time-to-time during training.
This is because GD with minibatch is a stochastic process and doesn't guarantee that the loss will decrease at each step.


setting up last layer in tensoflow for class type of label I am creating a NN in tensorflow keras. the inputs are all float and the output is a class.
The output currently encoded as a float, but only has 4 values (0,1,2,3).
My model is similar to this:
model = tf.keras.Sequential([
    normalize,
    layers.Dense(128, activation='relu'),
    layers.Dense(254, activation='relu'),
    layers.Dense(512, activation='relu'),
    layers.Dense(512, activation='relu'),
    layers.Dense(512, activation='relu'),
    layers.Dense(512, activation='relu'),
    layers.Dense(512, activation='relu'),
    layers.Dense(512, activation='relu'),
    layers.Dense(254, activation='relu'),
    layers.Dense(128, activation='relu'),
    layers.Dense(4)
])

model.compile(loss = tf.keras.losses.SparseCategoricalCrossentropy(), metrics=['accuracy'],
                           optimizer = tf.optimizers.Adam())

history=model.fit(data_set_features, data_set_labels,validation_split=0.33, epochs=100)

is the model last layer correct?
what type of activation should I use and loss function?

Since you're using categorical cross-entropy loss, the last layer (output layer) should come with softmax activation instead of identity (as being blank in layers.Dense(4)).
model = tf.keras.Sequential([
    normalize,
    ...,
    layers.Dense(4, activation=tf.keras.activations.softmax)
])

And SparseCategoricalCrossentropy is different from CategoricalCrossentropy, for categorical cross-entropy, the shape of output and label should be the same, for example:
When using categorical cross-entropy (example labels in one-hot):
Output: [[.1,.0,.9,.0],...] <-- Prediction of 90% for the 3rd class, sum=100%
Label:  [[ 0, 0, 1, 0],...] <-- Third class holds 100% classification possibility

When using sparse categorical cross-entry (labels are class indices):
Output: [[.1,.0,.9,.0],...] <-- Prediction of 90% for the 3rd class, sum=100%
Label:  [[2]          ,...] <-- Third class index is 2


Can we destroy an artificial general intelligence without its consent? According to Wikipedia Artificial general intelligence(AGI)

  Artificial general intelligence (AGI) is the intelligence of a
  (hypothetical) machine that could successfully perform any
  intellectual task that a human being can.

According to the below image (a screenshot of a picture from When will computer hardware match the human brain? (1998) by Hans Moravec, and Kurzweill also uses this diagram in his book The Singularity Is Near: When Humans Transcend Biology), today's artificial intelligence is the same as that of lizards.

Let's assume that within 10-20 years, we, humans, are successful in creating an AGI, that is, an AI with human-level intelligence and emotions.

At that point, could we destroy an AGI without its consent? Would this be considered murder?

Firstly, an AGI could conceivably exhibit all of the observable properties of intelligence without being conscious. Although that may seem counter-intuitive, at present we have no physical theory that allows us to detect consciousness (philosophically speaking, a 'Zombie' is indistinguishable from a non-Zombie - see the writing of Daniel Dennett and David Chalmers for more on this). Destroying a non-conscious entity has the same moral cost as destroying a chair.

Also, note that 'destroy' doesn't necessarily mean the same for entities with persistent substrate, i.e. their 'brain state' can be reversibly serialized to some other storage medium and/or multiple copies of them can co-exist. So if by 'destroy' we simply mean 'switch off', then an AGI might conceivably be reassured of a subsequent re-awakening. Douglas Hofstadter gives an interesting description of such an 'episodic consciousness' in "A Conversation with Einstein's Brain"

If by 'destroy', we mean 'irrevocably erase with no chance of re-awakening', then (unless we have a physical test which proves it is not conscious) destroying an entity with a seemingly human-level awareness is clearly morally tantamount to murder. To believe otherwise would be substrate-ist - a moral stance which may one day be seen as antiquated as racism.


Were there known tests done on two or more AI interacting together? I am talking about relationships between AIs (e.g. 2 of them forming a couple, 3+ in family like relationship).

What knowledge could come out of such experimentation?

One example might be self-play in games. Since neural networks and deep learning depend on massive amounts of data, one way to generate data is to have two virtual machines play each other and record the experience. An example discussion can be found at http://www.cs.cornell.edu/boom/2001sp/Tsinteris/gammon.htm which uses reinforcement learning. I believe AlphaGo also uses this technique of self-play, and uses two independent neural networks, one reducing the search space and the other deciding on the best move in the remaining space, that in a sense cooperate to decide on the next move.


Can Bayesian inference be combined with knowledge-based systems? I've been struggling with the connection between knowledge-based AI systems and Bayesian inference for a while now. While I continue to sweep through the literature, I would be happy if someone can answer these more specific questions directly

Are Bayesian inference based methods used in reasoning or Q/A systems -- to arrive at conclusions about questions whose answers are not directly present in the knowledge base?

In other words, if a Q/A system doesn't find an answer in a knowledge base, can it use Bayesian inference to use the available facts to suggest answers with varying likelihoods?

If yes, could you point me to some implementations?

Yes, it is possible to combine probabilistic / bayesian reasoning and a traditional "knowledgebase".  And some work along those lines has been done.  See, for example, ProbLog ("Probabilistic Prolog") which combines logic programming and probabilistic elements.  See:

https://dtai.cs.kuleuven.be/problog/tutorial/mpe/01_bn.html

Another project to look at is Pr-OWL
 ("Probabilistic OWL") which adds Bayesian reasoning to the Semantic Web stack.

Of course neither of these deals specifically with QA systems, but both represent some work on at least the foundational aspect of combining traditional logic and/or ontologies, with probabilistic approaches.  Building a QA system on top of that is an exercise for the reader...


Does it exist a human-like artificial intelligence? Does it exist a human-like artificial intelligence?

I define human-like as something that can act like a human in most aspects.

This depends on your definition of human-like.

If you mean a robot that looks and acts like a human, arguably, yes. Here's one of many examples: http://www.hansonrobotics.com/robot/sophia/

If you are looking for something that performs work and tasks, or works and thinks and talks like-or better than a human, the answer is mostly no, not yet.

I recommend you look at 'ANI, AGI, ASI"

ANI: artificial narrow intelligence. This is what you see around you right now.

AGI: artificial general intelligence. A theoretic AI that can "think" like a human. It does not yet exist. Estimates are between 20-60 years before we will successfully create AGI.

ASI: artificial super intelligence. In a nutshell, it is theorized to be everything we wish we could be or hope never to be. It does not exist yet. It is generally believed that, IF we create an AGI, ASI will evolve seconds or less than a decade after AGI is created.


Non-Convex Loss Function in Deep Learning Is a Big Deal? I want to use deep learning to estimate the value of a function based on some data. However, the loss function would be neither convex nor concave. Can I know if it is a big deal in deep learning? Is training a deep network, when loss function is convex, the same as optimizing a convex problem or not? I would be thankful if any paper has addressed this issue.

Optimization

In optimization, the loss function (sometimes called the error function) is a function that aggregates the disparity between actual and ideal behavioral states in multiple dimensions and over a sequence of input cases.  In re-entrant (reinforced) learning, a feedback scalar or vector acts as a corrective signal that can replace or further aggregate with the loss function and additionally impact the training back-propagation.

Convergence

In any of these architectures, including systems without NN components but attempt to adapt by converging on an optimal static or moving target behavior, one generality can be made.  As the goal state is approached, the probability of successful convergence increases if the size of each incremental estimation decreases.  The estimation is nothing more than an informed guess.

Decreasing the learning rate as the detected convergence value improves is one strategy being used experimentally if not in production, but that strategy has drawbacks when used alone.  Involving the loss function in slowing down as the destination approaches is a best practice.

Biology Analogy

A biological system of a similar nature is the human subjective experience of pain.  As the pain level goes down, the human brain cares less about the pain, therefore the steps taken to reduce it decrease and eventually vanish.  Evolution has proven such to be advantageous for the same reason.

The Mathematics Involved

Maximizing the probability of an unknown function to be learned adequately to perform well is done in NNs by convergence.  The term gradient descent is often used to describe the iterative process intended to converge on some ideal characterized by labeled data, a concept built into the network, or some fitness signal.  The likelihood of converging on at least a local minima (which may or may not be the global minima) is much higher if the slope decreases as the minima is approached in successive approximation scenarios.  This is when d2E / dt2 is positive.

The geometric idea of convexity is correlated to the calculus concept of the second derivative of a line or surface with respect to time or some other measure of forward progress.  In successive approximation, the independent variable that measures forward progress could be time, computing cycles, the index of the training sample, iteration number, or some aggregation of these.  The second derivative is the rate of the rate of change.

Convergence is more likely if the rate of change decreases as the disparity between what is perceived as optimal behavior and what is the actual current behavior.  In other terms, the relationship between risk in making the next adjustment to circuit (NN) behavior should approach zero as proximity to the optimal behavior approaches zero.  (If one senses they have worked there way to close proximity of their desired state, it makes no sense to make wild guesses.)

An Easy to Visualize Analogy

If you drop a rubber ball into a rigid cone, it will take time to reach thermodynamic rest, at the bottom.  A lossless ball (considered impossible) will bounce forever.  A paraboloid (parabolic in two dimensions like a solar reflector) will produce faster convergence with the same rubber ball because the ball drops in energy (the sum of kinetic and potential energy) with each bounce.  The trajectory does not overshoot the bottom nearly as much or as frequently.  This analogy is not perfect, but it provides a visual image without a diagram.

If you aggregate your disparity between your target trained behavior and the current in-training behavior in a way where the second derivative is negative (concave loss function with respect to distance) on either side of the targeted ideal, convergence is much less likely.  In the analogy, the rubber ball is likely to bounce out of the flared cone altogether.  A lossless ball will always bounce out eventually.

A more provincial analogy is that it would be like trying to catch a baseball with the back of a baseball glove.

Concave, Convex, or Zero Second Derivative

Whether continuous or discrete, convex functions converge much more frequently and usually with less time and computing resource than concave ones.  A second derivative of zero is in the middle.  The word linear is actually incorrect for this zero acceleration case.  The correct term is first degree polynomial.

Sum of squares over the dimensions of the domain (inputs) and over the sequence of input cases will perform well in many cases.  If you were to sum the square roots of the absolute value of error instead, your NN will rarely converge at all.

Executive Summary

The following three things are more likely to be favorable if you find a way to aggregate your disparity between ideal and actual current behaviors in a way where the second derivative is greater than zero.

Reliability of eventual convergence
Speed of convergence
Response time in the case of re-entrant (reinforced) learning
Savings of computing cycles
Reduction in the complexity of introspection
Conservation of computational memory (RAM or SDD)
Conservation of space needed for persistance and archiving
Reduction of project cost to the business


Aren't scores in the Wasserstein GAN probabilities? I am quite new to GAN and I am reading about WGAN vs DCGAN.
Relating to the Wasserstein GAN (WGAN), I read here

Instead of using a discriminator to classify or predict the probability of generated images as being real or fake, the WGAN changes or replaces the discriminator model with a critic that scores the realness or fakeness of a given image.

In practice, I don't understand what the difference is between a score of the realness or fakeness of a given image and a probability that the generated images are real or fake.
Aren't scores probabilities?

Figure 3 in the original WGAN paper is actually quite helpful to understand the difference between the score in WGAN and the probability in GAN (see screenshot below). The blue distribution are real samples, and the green one are fake samples. The Vanilla GAN trained in this example identifies the real samples as '100% real' (red curve) and the fake samples as '100% fake'. This leads to the problem of vanishing gradients and the well-known mode collapse of original GANs.
The Wasserstein GAN, on the other hand, gives each sample a score. The benefit of the score is that we can now identify samples that are more likely real than others, or more likely fake. For example, the further to the left a distribution is located, the more negative the WGAN score will be. We have therefore a continuum that doesn't end in 0 and 1, but can compare between samples that are 'good' and those which are 'better'. A normal GAN would identify both as 'good', making further improvement difficult.


Why does the Atari Gym Amidar environment only move after a certain number of episodes? When I try to run Amidar even without RL code, I cannot get the environment to move immediately. It takes about 100 steps before the game actually starts moving. I use the following simple code to display some images and print some actions (I always try to do the same action, namely going up):
env = gym.make('Amidar-v0')
env.reset()

for i in range(1000):
    action = 2
    next_state, reward, terminated, info = env.step(action) # take a random action
    print(f"Timestep {i}")
    print(next_state.shape)
    print(reward)
    print(action)
    print(info)
    plt.imshow(next_state)
    plt.show()

When running this code, it takes until about step 85 before the environment starts to move. After that, each step, it moves until the agent is hit by the enemy. Then the environment restarts in the start state, and it takes quite some time before it starts to move again. I have tried doing 'FIRE' as my first action; however, this is not working since it also takes a while before the environment starts moving. Because of this, my buffer is almost always filled with the same images and hence my network isn't learning anything. How do I get this environment to move immediately?

I reviewed some videos of Amidar on YouTube, and it seems that the game screen is fixed for a few seconds before the start of each level. This is probably to give the human player a chance to rest between levels and maybe even pause the game to take a break, as the gameplay looks very intense.

How do I get this environment to move immediately?

I suggest creating a gym wrapper to change the reset function of the environment to produce a different start state. Since the initial 85 frames are not influenced by the agent's actions, these frames are unhelpful and unnecessary for your agent's training. You could consider calling the step function 85 times in your new implementation of the reset function using a random action; then return the resultant state. When calling your newly implemented reset function, the agent's start state will be frame 85 instead of frame 0. As you noted, frame 85 is immediately followed with many frames of meaningful movement, which will provide higher-quality training data. In contrast, the original implementation of the reset function starts at frame 0, which is followed by 85 unhelpful fixed frames.

Because of this, my buffer is almost always filled with the same images and hence my network isn't learning anything.

ATARI environments are difficult even for some of the more powerful RL algorithms. For example, it is one of the harder environments for DQN to solve (see the Nature paper, pg. 531).  Don't worry if your agent isn't learning right away. It could take hours or even days to learn enough to noticeably improve, even with a correct implementation of a powerful algorithm. Good luck, and welcome to AI Stack Exchange!


What is the right way to train a generator in a GAN? I am not fully understanding how to train a GAN's generator. I have a few questions below, but let me first describe what I am doing.
I am using the MNIST dataset.

I generate a batch of random images (the faked ones) with the generator.

I train the discriminator with the set composed of faked images and real MNIST images.

After the training phase, the discriminator modifies the weights in the direction of recognizing fake (probability 0) from real (probability 1) ones.

At this point, I have to consider the combined model of generator and discriminator (keep untrainable the discriminator) and put in the generator as input the faked images with the tag of 1s (as was real one).

My questions are:
Why do I have to set to real these fake images, and what fake images are these? The one generated in the first round from the generator itself? Or only the one classified as faked by the discriminator? (Then they could be both real images classified wrongly or fake images classified in the right way). Finally, what the generator does to these faked images?

Why I have to set to real these fake images and what fake images are these?

You set them to "real" label for the discriminator when training the generator, because that is the goal of the generator, to produce an output of 1 (probability of being a real image) when tested.
Usually you will generate a new batch of generated images for this step in training. You just used the last generated mini-batch to train the discriminator, so you expect them to score worse. Sending the exact same images again will cause correlation between the two minibatches that you want to avoid. It would not be a disaster, but training GANs can be quite difficult and sensitive to details like this, so it is better to keep generating new images and not re-use the previous ones.

The one generated in the first round from the generator itself?

No. New images generated just for training the generator.

Or only the one classified as faked by the discriminator? (then they could be both real images classified wrongly or fake images classified in the right way).

No. New images generated just for training the generator.
Out of interest though, if the discriminator classifies a fake image as 100% real (with a probability close to 1), then the generator will not learn anything from that. The gradients would all be zero.

Finally what the generator does to these faked images?

Nothing is done to the images themselves - unless perhaps you are keeping some copies to render and monitor training progress etc. The images occur within the combined generator/discriminator network, effectively as a hidden layer. The images are represented as artificial neuron output, so they are involved in backpropagation calculations for that layer (with no difference to any other hidden layer in a CNN), but are not changed directly.
The generator uses the gradients calculated from the combined discriminator/generator network to update its weights using gradient descent. Importantly in this phase of the updates, the discriminator weights are not changed.
In terms of training the generator/discriminator combined network to update the generator:

The input to the combined network is some new random input vectors (typically a vector with independent truncated normal distribution for each element).

The "ground truth" label is 1 for every item.

The discriminator parameters must be "frozen", somehow excluded from being updated.

Run the minibatch forward to get loss and backpropagate to get gradients for the whole network including the generator.

Apply a gradient step (usually via some optimiser, such as Adam).


Could GA's determine fitness by "Fighting" against each other? I am developing AI in the form of NEAT, and it has passed certain tasks like the XOR problem outlined in the NEAT Research Paper. In the XOR Problem, the fitness of a network was determined by an existing function (XOR in this case). It also passed another tests. One I developed was to determine the sine at a certain point X in radians. It also worked, but yet again, its fitness was determined by an existing function (sin (x)).

I've recently been working on training it to play Tic Tac Toe. I decided that to determine its fitness, it would play against a "dumb" AI, placing O's in random locations on the grid, and gaining fitness based on whether or not it placed X's in a valid location (losing fitness if it placed an X on top of another X or an O), and gaining a lot of fitness if it won against the "dumb" AI. This would work, but when a network got really lucky and the "dumb" AI placed O's in impractical locations, the network would win and gain a lot of fitness, making it very difficult for another network to beat that fitness. Therefore, the learning process did not work and I was not able to generate a Tic Tac Toe network that actually worked well.

I do not want the GA to learn based off an "intelligent" tic tac toe AI because the whole point of me training this GA is so that I do not have to make the AI in the first place. I want it to be able to learn rules on its own without me having to hard code an AI to be very good at it.

So, I got to thinking, and I thought it would be interesting if the fitness of a network could be determined based off how well it played against OTHER NETWORKS in its generation. This does seem similar to how humans learn to play games, as I learned to play chess by playing against other people hundreds of times, learning from my mistakes, and my friends also increased in their ability to play chess as well. If GA's were to do that, that would mean I don't have to program AI to play the game (in fact, I wouldn't have to program a "dumb" AI as well, I would only have to hard code the rules of the game, obviously).

My questions are:

Has there been any research or results from GA's determining their fitness based off competing against each other? I did some searching but I have no idea what to look for in the first place (searching 'NEAT fight against each other' did not work well :-(  )
Does this method of training a GA seem practical? It seems practical to me, but are there any potential drawbacks to this? Are GA's meant to only calculate predetermined functions that exist, or do they have the potential to learn and do some decision making?
If I were to do this, how would fitness be determined? Say, for the tic tac toe example, should fitness be determined based on whether or not a network places its X's or O's in viable locations, and add fitness if it wins and subtracts fitness if it loses? What about tying the game?
Should networks of the same species compete against each other? If they did, then it would seem impractical to have species in the first place, as networks in the same species competing against each other would not allow a successful species to rise to the top, as it would be fighting against each other.
Kind of out of topic, but with my original idea for the tic tac toe GA, would there be a better way to determine fitness? Would creating an intelligent AI be the best way to train a GA?

Thanks for your time, as this is somewhat lengthy, and for your feedback!

i'm the main developer of Neataptic, a Javascript neuro-evolution library.

Very effective! Realise that this is how real-life evolution happened as well: we kept on improving against other species, which forced them to improve as well.
Very practical, especially if you don't want to set up any 'rules' like you say, it makes the genomes find out what the rules are themselves.
Basically, you let each genome in the population play X games against other genomes, I advise you let each genome play against every other genome in the population. An example of scoring would be giving the genome 1 point for winning, and 0.25 or 0.5 for a tie. Each game should always have a result!
I'm not sure about this one, as I haven't implemented speciation.

I want to give you some examples that I have worked on:

Agar.io AI (neuro-evolved neural agents) - basically, I let neural networks evolve to get the highest score they can in agar.io, by competing against each other! It worked better than I expected.
Currently i'm working on new project, a kind of 'cops and robbers' style game.


What is the concept of the technological singularity? I've heard the idea of the technological singularity, what is it and how does it relate to Artificial Intelligence? Is this the theoretical point where Artificial Intelligence machines have progressed to the point where they grow and learn on their own beyond what humans can do and their growth takes off?  How would we know when we reach this point?

The technological singularity is a theoretical point in time at which a self-improving artificial general intelligence becomes able to understand and manipulate concepts outside of the human brain's range, that is, the moment when it can understand things humans, by biological design, can't.
The fuzziness about the singularity comes from the fact that, from the singularity onwards, history is effectively unpredictable. Humankind would be unable to predict any future events, or explain any present events, as science itself becomes incapable of describing machine-triggered events. Essentially, machines would think of us the same way we think of ants. Thus, we can make no predictions past the singularity. Furthermore, as a logical consequence, we'd be unable to define the point at which the singularity may occur at all, or even recognize it when it happens.
However, in order for the singularity to take place, AGI needs to be developed, and whether that is possible is quite a hot debate right now. Moreover, an algorithm that creates superhuman intelligence (or superintelligence) out of bits and bytes would have to be designed. By definition, a human programmer wouldn't be able to do such a thing, as his/her brain would need to be able to comprehend concepts beyond its range. There is also the argument that an intelligence explosion (the mechanism by which a technological singularity would theoretically be formed) would be impossible due to the difficulty of the design challenge of making itself more intelligent, getting larger proportionally to its intelligence, and that the difficulty of the design itself may overtake the intelligence required to solve the said challenge.
Also, there are related theories involving machines taking over humankind and all of that sci-fi narrative. However, that's unlikely to happen, if Asimov's laws are followed appropriately. Even if Asimov's laws were not enough, a series of constraints would still be necessary in order to avoid the misuse of AGI by misintentioned individuals, and Asimov's laws are the nearest we have to that.


Who first coined the term "artificial general intelligence"? Similarly to the question Who first coined the term Artificial Intelligence?, who first coined the term "artificial general intelligence"?

According to Ben Goertzel, the first person that probably used the term "artificial general intelligence" (in an article related to artificial intelligence) was Mark Avrum Gubrud in the 1997 article Nanotechnology and International Security. Here's an excerpt from the article.

  By advanced artificial general intelligence, I mean AI systems that rival or surpass the human brain in complexity and speed, that can acquire, manipulate and reason with general knowledge, and that are usable in essentially any phase of industrial or military operations where a human intelligence would otherwise be needed. Such systems may be modeled on the human brain, but they do not necessarily have to be, and they do not have to be "conscious" or possess any other competence that is not strictly relevant to their application. What matters is that such systems can be used to replace human brains in tasks ranging from organizing and running a mine or a factory to piloting an airplane, analyzing intelligence data or planning a battle.

Note that the term "AGI" could have been used even before that Mark A. Gubrud's article (as Goertzel also suggested).

In any case, Ben Goertzel help to popularise this term, especially with the book Artificial General Intelligence. AGI was previously known as "strong AI", which goes back to John Searle's Chinese Room argument, although strong AI often refers to an AGI with consciousness, and the definition of AGI doesn't necessarily imply consciousness.

You can read more about the history of the term "AGI" in this Goertzel's blog post.


Regression For Elliptical Curve Public Key Generation Possible? As part of a learning more about deep learning, I have been experimenting with writing ResNets with Dense layers to do different types of regression.
I was interested in trying a harder problem and have been working on a network that, given a private key, could perform point multiplication along ECC curve to obtain a public key.
I have tried training on a dataset of generated keypairs, but am seeing the test loss values bounce around like crazy with train loss values eventually decreasing after many epochs due to what I assume is overfitting.
Is this public key generation problem even solvable with a deep learning architecture? If so, am I doing something wrong with my current approach?

I don't know of any neural network that can do cryptography well, so you would have to do a little experimenting yourself. The main thing that sticks out to me is that doing operations in the elliptical curve requires the modulus operator since it works in finite field, and I think neural networks have a hard time learning the modulus operator in general. So I would focus on that first. Some things to help the network learn the modulus operator:

I would try to increase the hidden layers to a number 4x-10x bigger than the input dimension, which maps it to a higher dimension to hopefully learn more complex behavior.
I would use less layers, maybe only 2-4 hidden layers, to speed up the development time.
Most importantly, I would train with a LOT of data points (25% of the total possible finite field possibilities). I don't think the neural network can learn unless the number of data is this high.
For reference, someone got a network to learn the modulus operator using these points.

For rapid iterating, I would test with a much smaller finite field. E.g. use a 8 bit security and see if the neural network can do well with that first before moving on to the full 256-bit security key (or whatever your end goal is).
Taking this a step further, I would first test to see if the neural network can even perform a point addition in the elliptical curve well, because if it doesn't then it definitely can't do point multiplication which is needed to compute the public key.


Are there any refutation of the original Lovelace Test? The original Lovelace Test, published in 2001, is used generally as a thought experiment to prove that AI cannot be creative (or, more specifically, that it cannot originate a creative artifact). From the paper:

Artificial Agent A, designed by H, passes LT if and only if

A outputs o,

A outputting o is not the result of a fluke hardware error, but rather the result of processes A can repeat

H (or someone who knows what H knows, and has H's resources) cannot explain how A produced o.

The authors of the original Lovelace Test then argues that it is impossible to imagine a human developing a machine to create an artifact...while also not knowing how that machine worked. For example, an AI that uses machine learning to make a creative artifact o is obviously being 'trained' on a dataset and is using some sort of algorithm to be able to make predictions on this dataset. Therefore, the human can explain how the AI produced o, and therefore the AI is not creative.
The Lovelace Test seems like an effective thought experiment, even though it appears to be utterly useless as an actual test (which is why the the Lovelace Test 2.0 was invented). However, since it does seem like an effective thought experiment, there must be some arguments against it. I am curious to see any flaws in the Lovelace Test that could undermine its premise.

I am a future neurologist with a very complete understanding of linguistic processing in the brain.  I am also an overprotective parent, so I monitor every phrase uttered to my child, and also completely determine all the books she reads in the course of her education.

When my child writes a poem, then, I know the dataset on which her brain was trained, as well as the processes by which her language inputs became language outputs--in broad outline I know these processes are non-linear and are based on how different inputs along with the current collection of trillions of distinct synaptic weights updates the synaptic weights.  I don't know what her poem will be, of course, because there are random factors and the whole history of her synaptic weights are unobservable, but I adhere to the Lovelace test and can therefore conclude that composing the poem was not a creative act.

The Lovelace Test, like the Chinese Room Argument, implicitly assumes that what computers/AI can do in processing symbols and information and what brains can do are distinct.  If you accept that assumption, then the argument ceases to be interesting-- you've merely redefined creativity as one of the distinct things that brains can do.  If you reject the assumption, the argument that computers are incapable of creativity ceases to be valid.  The thought experiment itself does nothing to assist us in evaluating the truth of the assumption.


Can a variational auto-encoder learn images composed of random noise at each pixel (each drawn from the same distribution)? Can a variational auto-encoder (VAE) learn images whose pixels have been generated from a Gaussian distribution (e.g. $N(0, 1)$), i.e. each pixel is a sample from $N(0, 1)$?

My gut feeling says no, because the VAE adds additional noise $\epsilon$ to the original image in the latent space, and if all images and pixels are random from the same distribution it would be impossible do decode/reconstruct into the particular input image. However, VAEs are a bit of a mystery to me internally. Any help would be appreciated!

VAE's try and model the distribution of your data. So it's not going to learn " images composed of random noise at each pixel" per se (though, if overfitting, it could remember them). But it would be very capable of learning the simple noise distribution from which you sampled your random pixels


Which technologies should humanity develop before developing an AI with general superhuman intelligence? The sequence in which new technologies are developed can be crucial for it's success and it's safe use. For example, before developing a nuclear reactor you want to have technologies like reliable cooling systems and secure ultimate disposal zones available and tested. Otherwise we might have a bad time.

I see AI with general superhuman intelligence as important predecessor for dangerous technologies like nanobots or generators using nuclear fusion, because superhuman intelligence can be helpful in foreseeing dangers that might lead to catastrophe using these technologies in an insecure way.

But what about important technologies we want to have before inventing an AI with general superhuman intelligence? Solving the control-problem is obvious, but are there other technologies we should have in advance?

I think most answers that come to mind are going to be connected to the control problem in some way---e.g. sandboxing frameworks, AI behavior monitoring protocols, etc.

However, if you aren't interested in those for now, that pretty much leaves two possible needs such technology could satisfy: the technical challenges of AI itself, and the need to optimize the social impact on AI.

The technologies satisfying the first need are relatively "obvious" once you have the AI implementation figured out---depending on what that is, there will be certain hardware and software requirements (massively parallel processing, neural network frameworks, etc). Most of these technologies seem likely to arise "naturally" from the status quo---if we just keep developing our hardware and software as we are now, we will eventually get there.

Coming up with technologies satisfying the second need is a challenge, and far more interesting in my opinion. We need to start a discussion about this, so you have a great question here. I don't know that I can contribute much, but maybe I can provide some starter ideas:

More robust encryption schemes, so that the massive amount of information handled by AI is secure even in the face of AI-capable adversaries. We'd want these encryption schemes to be information-theoretic secure, so that it is mathematically impossible to recover the plaintext, rather than just computationally difficult, since an AI may be able to improve it's own computational efficiency with unprecedented ease. Quantum cryptography may help.
Improved robotics, which would require improved battery technology, to multiply by a great factor the ability of AI to manipulate the physical world and assist with physical tasks (obviously requires a fantastic solution to the control problem).
If you consider it separate from the control problem (and I think there is a useful distinction here), lots of AI-monitoring technology will be necessary. Think of technologies acting as regulators for newly automated tasks---e.g. traffic monitoring algorithms to watch for undesirable behavior among automated vehicles.

Answering this question is very tough, particularly because you are asking for technologies which we'd have to develop before we have AI. Of course, once we have general AI we can instruct it to develop any of these technologies for us, supposing the control problem is adequately solved. Nevertheless it seems wise to solve develop improved cryptography immediately since even with weak AI (or a weak control problem solution) the need is urgent.


Are my steps correct for a proper classification of a sick brain? I have a dataset with MRI of patients with a specific disease that affects the brain and another dataset with MRI of healthy patients.

I want to create a classifier (using neural networks) to classify if the MRI of a new patient shows the presence of the illness or not.

First of all, I extracted the brain from all the MRIs (the so-called skull stripping) using BET tool found in FSL.

I have three questions for you

As the input to the training phase, I want to give the whole extracted brains (possibly in the nii format). What kind of preprocessing steps do I need to apply once I've extracted the brains (before passing it to the classifier)?
Do you know any better tool for skull stripping?
Do you know a tool (or library) that takes as input a nii files and allows me to create a classifier that uses neural networks?

It looks like everything you want is available with the Deep Learning Toolkit (DLTK) for Medical Imaging

There is also a blog: An Introduction to Biomedical Image Analysis with TensorFlow and DLTK

There is a DataCamp course that walks you through most of the process but instead of a classifier they use deep learning to reconstruct brain images.  They provide a link to their MNIST classifier example which should be easy to adapt for your purpose.  See: Reconstructing Brain MRI Images Using Deep Learning (Convolutional Autoencoder)

ResearchGate has a thread that may help:
What is the appropriate way to use Nifti files in deep learning?


How can the policy iteration algorithm be model-free if it uses the transition probabilities? I'm actually trying to understand the policy iteration in the context of RL. I read an article presenting it and, at some point, a pseudo-code of the algorithm is given :

What I can't understand is this line :

From what I understand, policy iteration is a model-free algorithm, which means that it doesn't need to know the environment's dynamics. But, in this line, we need $p(s',r \mid s, \pi(s))$ (which in my understanding is the transition function of the MDP that gave us the probability of landing in the state $s'$ knowing previous $s$ state and the action taken) to compute $V(s)$. So I don't understand how we can compute $V(s)$ with the quantity $p(s',r \mid s, \pi(s))$ since it is a parameter of the environment.

Everything you say in your post is correct, apart from the wrong assumption that policy iteration is model-free. PI is a model-based algorithm because of the reasons you're mentioning.

See my answer to the question What's the difference between model-free and model-based reinforcement learning?.


What is the current state of AGI development? Could you please provide some insight into the current stage of developments in AGI area? Are there any projects that had breakthroughs recently? Maybe some news source to follow on this topic?

The state of AGI research is pursuing the few problems that we have been able to break off from the gigantic research problem. These are terms which can be more thoroughly looked into.

A few of the main focuses are:

One Shot learning - You know how a person can sometimes learn to do something by something by seeing literally 1 example of it? Well current learning methods on the whole are not able to accomplish this to the extent that we can easily take for granted. Work is being done to find ways to approach this feat of learning and it’s on its way to becoming much more influential.
Transfer Learning - If you have ever played a side scroller like Mario, and then I gave you a slightly different game like Sonic, odds are you’d could learn to play Sonic faster than it took you to learn to play Mario. This is because of learning “savings” you have by transferring your Mario knowledge to the new Sonic domain. This is a much more popular research arm than one-shot, probably because it’s easier to think about but also because there have been promising results of pretraining a network on one set of data and focusing it an another task.
Creativity/curiosity - Although one could say that GANs have really changed how humans can be more creative, it is difficult to quantify curiosity and creativity This paper gives an okay overview. Moreover, allowing an agent to take the chances and make some mistakes as is the nature of creativity concerns many people who are focused on AI Safety.
Understanding concepts - This is subtle but very very important. Current AI methodologies struggle with imbuing AI with the ability to have concepts. By concepts, I don’t mean “it kinda looks like this neuron in the second last layer is sensitive to tires”. I mean that understanding what a tire looks like is just a small part of understanding what a tire is, what it is used for, what it affords someone to do etc. This research direction is in it’s infancy but will be much more influential as more theories and ideas are brought forward.

Despite the progress made in these fields and in the many other areas in AI, there is still much to be done and understood before we can finally have s̶k̶y̶n̶e̶t̶ Wall-E.


How to formalize learning in terms of information theory? Consider the following game on a MNIST dataset:

There are 60000 images.
You can pick any 1000 images and train your Neural Network without access to the rest of images.
Your final result is prediction accuracy on all dataset.

How to formalize this process in terms of information theory? I know that information theory works with distributions, but maybe you can provide some hints how to think in terms of datasets instead of distributions.

What is the information size of all datasets. My first idea was
that each image is iid from uniform distribution and information
content = -log2(1/60000). But common sense and empirical results
(training neural network) show that there are similar images and
very different images holding a lot more information. For example if
you train NN only on good looking images of 1 you will get bad
results on unusual 1s.
How to formalize that the right strategy is to choose as much as possible different 1000 images. I was thinking to take image by image with the highest entropy relative to the images you already have. How to define distance function.
How to show that all dataset contains N bits of information, training dataset contain M bits of information and there is a way to choose K images < 60000 that hold >99.9% of information.

In short: It is easy to quantify information, but it is not easy to quantify its usefulness

I'm not sure how exactly you are looking to formalise your experiment, but it might be helpful to consider these points:

There is no such thing as an absolute measure of information. The amount of information contained in some dataset is dependent on the underlying assumptions that are made when interpreting it, and therefore, the quantity of information conveyed is also dependent on the encoder/decoder (for example, a neural network). See the Wikipedia article on Kolmogorov Complexity.
Entropy is a useful measure of information content when you assume each sample is iid, but this would be a very bad assumption to make for natural images, since they are highly structured. For example, imagine an image with 50% black pixels and 50% white pixels that can be arranged in any configuration - not matter how you arrange them, wether it looks like random noise, a text paragraph, or a chequer board, the entropy value will be identical for each, even though our intuition tells us otherwise (see attached image). The discrepancy between our intuition and the entropy value arises because our intuition does not interpret the image through the "lens" of iid pixels, but rather, hierarchical receptive fields in the visual cortex (somewhat analogous to convolutional neural networks).
Calculating the entropy of pixel values in one image is somewhat useful, but calculating the "entropy" of a set of images would not be useful, because each image as a whole is treated as if it were a unique arbitrary symbol. I assume this is what you meant by "the information size of all datasets"
KL-divergence is a distance function that is often used to compare two distributions. Intuitively, it represents the redundant bits generated by a non-ideal compression program that assumes an incorrect data distribution. However, KL-divergence between two natural images will not give you a particularly meaningful result.

If I am not mistaken, you want to find some information metric that will enable you to pick the smallest number of the most optimal images for training and get a good test score with the network. Is that correct? It is an interesting idea. However, in order to define such a metric, we might have to know in advance what features of an image are the most significant for classification, which in some ways defeats the point of using machine learning in the first place, where non-obvious and "hidden" features are exploited.


Is it possible to train a neural network as new classes are given? I would like to train a neural network (NN) where the output classes are not (all) defined from the start. More and more classes will be introduced later based on incoming data. This means that, every time I introduce a new class, I would need to retrain the NN.
How can I train an NN incrementally, that is, without forgetting the previously acquired information during the previous training phases?

I'd like to add to what's been said already that your question touches upon an important notion in machine learning called transfer learning. In practice, very few people train an entire convolutional network from scratch (with random initialization), because it is time consuming and relatively rare to have a dataset of sufficient size.

Modern ConvNets take 2-3 weeks to train across multiple GPUs on ImageNet. So it is common to see people release their final ConvNet checkpoints for the benefit of others who can use the networks for fine-tuning. For example, the Caffe library has a Model Zoo where people share their network weights.

When you need a ConvNet for image recognition, no matter what your application domain is, you should consider taking an existing network, for example VGGNet is a common choice.

There are a few things to keep in mind when performing transfer learning:

Constraints from pretrained models. Note that if you wish to use a pretrained network, you may be slightly constrained in terms of the architecture you can use for your new dataset. For example, you can’t arbitrarily take out Conv layers from the pretrained network. However, some changes are straight-forward: due to parameter sharing, you can easily run a pretrained network on images of different spatial size. This is clearly evident in the case of Conv/Pool layers because their forward function is independent of the input volume spatial size (as long as the strides “fit”).
Learning rates. It’s common to use a smaller learning rate for ConvNet weights that are being fine-tuned, in comparison to the (randomly-initialized) weights for the new linear classifier that computes the class scores of your new dataset. This is because we expect that the ConvNet weights are relatively good, so we don’t wish to distort them too quickly and too much (especially while the new Linear Classifier above them is being trained from random initialization).

Additional reference if you are interested in this topic: How transferable are features in deep neural networks?


Ideal score of a model on training and cross validation data The question is little bit broad, but I could not find any concrete explanation anywhere, hence decided to ask the experts here.

I have trained a classifier model for binary classification task. Now I am trying to fine tune the model. With different sets of hyperparameters I am getting different sets of accuracy on my train and test set. For example:

(1) Train set: 0.99 | Cross-validation set: 0.72
(2) Train set: 0.75 | Cross-validation set: 0.70
(3) Train set: 0.69 | Cross-validation set: 0.69

These are approximate numbers. But my point is - for certain set of hyperparameters I am getting more or less similar CV accuracy, while the accuracy on training data varies from overfit to not so much overfit.

My question is - which of these models will work best on future unseen data? What is the recommendation in this scenario, shall we choose the model with higher training accuracy or lower training accuracy, given that CV accuracy is similar in all cases above (in fact CV score is better in the overfitted model)?

You should only look for the cross-validation score. If this set is large enough, it will give you an accurate prediction of how your model will act for unseen data.

Your case is exceptional. The fitted model which is obviously overfitted actually performs better on the cross-validation set. This means in turn that your overfitted model will perform better with unseen data.


Should I prefer cropped images or realistic images for object detection? I am new to the field of AI but due to the high level of abstraction that comes with services such as Google VisionAI I got motivated to write an application that detects symbols in photos based on tensorflow.js and a custom model trained in Google Vision AI.
My App is about identifying symbols in photos, very similar to traffic signs or logo detection. Now I wonder if

I should train the model based on real, distorted and complex photos that contain those symbols and lots of background noise
if it was enough to train the model based on cropped, clean symbols
A hybrid of both

I started with option a and it works fine, however it was a lot of work to create the training dataset. Does the model need the distorted background to work?

Google recommandation seems to answer this:

The training data should be as close as possible to the data on which
predictions are to be made.
For example, if your use case involves blurry and low-resolution
images (such as from a security camera), your training data should be
composed of blurry, low-resolution images. In general, you should also
consider providing multiple angles, resolutions, and backgrounds for
your training images.

https://cloud.google.com/vision/automl/object-detection/docs/prepare
Would you agree, also in the case of symbols?


Dataset for floating objects detection I am looking for a dataset, which I could train a model to detect people/boats/surfboards, etc., from a drone view.
Has anyone seen a dataset that could be useful for this purpose?
I have some photos made by me (like this one below), but I need more data. Of course, the best will be if data will be labeled, but, if someone has seen an unlabeled dataset with videos/photos like that below, please share the link to it.
Sample photos I am looking for:

Perhaps you can check this dataset out:
http://www.aiskyeye.com/

  The VisDrone2019 dataset is collected by the AISKYEYE team at Lab of
  Machine Learning and Data Mining , Tianjin University, China. The
  benchmark dataset consists of 288 video clips formed by 261,908 frames
  and 10,209 static images, captured by various drone-mounted cameras,
  covering a wide range of aspects including location (taken from 14
  different cities separated by thousands of kilometers in China),
  environment (urban and country), objects (pedestrian, vehicles,
  bicycles, etc.), and density (sparse and crowded scenes). Note that,
  the dataset was collected using various drone platforms (i.e., drones
  with different models), in different scenarios, and under various
  weather and lighting conditions. These frames are manually annotated
  with more than 2.6 million bounding boxes of targets of frequent
  interests, such as pedestrians, cars, bicycles, and tricycles. Some
  important attributes including scene visibility, object class and
  occlusion, are also provided for better data utilization.

It provides many drone view images with bounding boxes. Hope it can help you and have a nice day!


Unsupervised learning to optimize a function of the input I am looking to build a neural network that takes an input vector $\mathbf{X}$ and outputs a vector $\mathbf{Y}$ such at $f(\mathbf{X}, \mathbf{Y})$ is minimized, where $f$ is some function. The network will see many different $\mathbf{X}$ during training to adjust its weights and biases; then I will test the network by using the test set $\{x_1, \dots, x_n \}$ to calculate $\sum(f(x_1, y), \dots, f(x_n, y))$ to see if this sum is minimized.

However, I have no labels for the output $\mathbf{Y}$. The loss function I am trying to minimize is based on the input and output instead of the output and label. I tried many standard Keras and TensorFlow loss functions, but they are unable to do the job. Any thoughts on how this might be achieved?

According to your description, you already know your function $f$ to be optimized. So you should use it directly instead of the standard loss functions. In this other post there is an explanation of how to use $f$ as a custom loss function in Keras.


The analysis of the dynamic behaviour of neural networks involving the application of feedback I am reading the Simon Haykin's cornerstone book, "Neural Networks, A Comprehensive Foundation, Second Edition" and I cannot understand a paragraph below:

  The analysis of the dynamic behaviour of neural networks involving the
  application of feedback is unfortunately complicated by virute (or
  virtue I cannot get word appropriately) of the fact that the
  processing units used for the construction of the network are usually
  nonlinear. Further consideration of this issue is deferred to the
  latter part of the book.

Before the paragraph, the author analysis the affects of weight of synapsis to the neural network's stability. Roughly speaking, he says, if |w| >= 1 the neural network become unstable.

Could you please explain the paragraph? Thanks in advance.

Although you have not given the entire context but if I were to speculate I would suggest the author is simply trying to tell that Neural Networks are quite difficult to analyse mathematically due to their non-linear nature (due to the use of non-linear activation). There are many questions on this Stack about the mathematical basis of NN.

Do scientists know what is happening inside artificial neural networks?

Also forward propagation is difficult enough, and now we that we have included feedback it becomes exponentially tough. You can easily understand the difficulty posed in such analysis if you take the case of Feedback Amplifiers in electronics. In normal sound amplifiers to avoid distortion we use a negative feedback. This roughly has an inverted waveform compared to i/p signal. It adds up with the input signal and smoothens out the distortions (as opposite wavefroms of not entirely same magnitude adds up).

|W| > 1 should be the case of Positive feedback, which can cause massive variations in output due to small changes due to its self reinforcing property. The wave-forms are of the same shape and they add up, this results in an even more larger waveform in the next cycle and so on. So probably the author is speaking something along these lines.


Identify "actionable" intents in email messages I am trying to build a model which would take an email message (in English, extracted subject, and body of the email) and identify if it has a question, request or a proposal. Basically, I would like to see the mails that I've not replied but needs a reply. The model can be used as a "filter" in an email client.

What is the best way to go about it?

Related Work:

Parakweet Lab's Email Intent Data Set

Learning to Classify Email into "Speech Acts"

This is a categorization problem, not unlike a spam filter. Instead of flagging an email as spam/not-spam, you are flagging whether it has one of the action categories that you have described.

You'll need to start by assembling a training corpus of example email messages and labeling each example to identify which (maybe multiple) of your categories, if any, are actually present in that email.

Next, pre-process that data to extract features for each message. Examples of typical features include word (or n-gram) counts/frequencies (bag of words). As a shortcut, you might to include as a feature a boolean indicating the presence or absence of a particular word or phrase that you suspect will be predictive of one or more categories. Techniques such as stemming can help reduce the number of words/n-grams being used (often increasing accuracy).

Once you have a dataset that consists of features and labels for each training email (possibly breaking this set up into subsets for training, cross-validation, and testing), you'll want to apply a supervised classification algorithm. You might start with linear classifiers such as logistic regression or SVMs, and if you're unsatisfied with the resulting accuracy then you could advance to neural techniques.


Can maximum likelihood be used as a classifier? I am confused in understanding the maximum likelihood as a classifier. I know what is Bayesian network and I know that ML is used for estimating the parameters of models. Also, I read that there are two methods to learn the parameters of a Bayesian network: MLE and Bayesian estimator.

The question which confused me are the following.

Can we use ML as a classifier? For example, can we use ML to model the user's behaviors to identify the activity of them? If yes, How? What is the likelihood function that should be optimized? Should I suppose a normal distribution of users and optimize it?
If ML can be used as a classifier, what is the difference between ML and BN to classify activities? What are the advantages and disadvantages of each model?

If you read nothing else, maximum likelihood estimate => chance that the data predicted is the data observed. If you have a range of points (2, 3, 4, 5, 71) your MLE is going to favour ~4.5 because of means and standard deviations. MLE speeds up finding good input parameters, usually for a different classifier.

To answer your question:

1) Columbia University have a great example of using MLE classifiers, where everything is broken down into bitesize (or bytesize) chunks. Read this. Seriously.

2) In short, MLE is best used for simple, univariable distributions. It doesn't scale well to big problems but is waaay faster than a Bayesian network for simple tasks like predicting your height based on the heights of your immediate relatives. If you want to get technical, the conditional probability network of the Bayesian model reveals insights faster than the chain multiplication of the more primitive MLE.

Hope it helps!


What is the difference between automatic transcription and automatic speech recognition? What is the difference between automatic transcription and automatic speech recognition? Are they the same?

Is my following interpretation correct?

Automatic transcription: it converts the speech to text by looking at the whole spoken input

Automatic speech recognition: it converts the speech to text by looking into word by word choices

They are both the same. There are different algorithms to recognise speech, but essentially they all aim to identify the content of the spoken input and convert it into written text.

Automatic transcription is then done, whereas the output of more general ASR is often passed on to further processing, such as recognising entities or commands expressed in the speech.


Why are "Transformers" called this way? What is the reason behind the name "Transformers", for Multi Head Self-Attention-based neural networks from Attention is All You Need?
I have been googling this question for a long time, and nowhere I can find any explanation.

The authors of the original paper don't provide an explanation, but I suspect it's a combination of:

popular recognizable branding (cf. BERT, DALL-E, Watson etc)
similarity to [sequence] transduction / translation / transformations generally


Why do I get the best policy before Q values converge using DQN? I have implemented DQN algorithm and wonder why during testing, the best performance is achieved by a policy from about 300 episode, when mean Q values converge at about 800 episode?

Mean Q-values are calculated on a fixed set of states by taking mean of max Q-values for each state.
By convergence I mean that the plot of mean Q-values converge to some level (those values does not increase to infinity).

It can be seen in here (page 7) that mean Q-values converge and average rewards plot is quite noisy. I get similar results and in tests, the best policy is where the peaks are during training (average reward plot). I don't understand why don't I get better average scores over time (and better policies) when Q-values converge.

Even if the mean of the maximum Q-value increases from episode 300 onwards, it doesn't mean that the relative order of the Q-values of the actions that you can take in the states change, which means that the policy may not change, even though the value function changes, assuming you're acting greedily with respect to the value function.
More concretely, suppose that you can take one of two actions $\{ a_1, a_2\} = \mathcal{A}$ in each state $s \in \mathcal{S}$. Let's say that you pick $s_1, s_2$ to calculate the average of the maximum Q-value. Without loss of generality, suppose that the action associated with the highest Q-value in these states is $a_2$. If your policy is the greedy policy with respect to the state-action value function, then your policy will choose $a_2$ in $s_1$ and $s_2$. If all Q-values $\hat{q}(s_1, a_1)$, $\hat{q}(s_1, a_2)$, $\hat{q}(s_2, a_1)$ and $\hat{q}(s_2, a_2)$ increase (or even decrease) but their relative order remains the same (i.e. $\hat{q}(s_1, a_2) > \hat{q}(s_1, a_1)$ and $\hat{q}(s_2, a_2) > \hat{q}(s_2, a_1)$, from episode 300 onwards), the greedy policy also remains the same.
So, I think that what you observed is theoretically possible, although I cannot guarantee that you don't have another issue.


Understanding LSTM through example I want to code up one time step in a LSTM. My focus is on understanding the functioning of the forget gate layer, input gate layer, candidate values, present and future cell states.
Lets assume that my hidden state at t-1 and xt are the following. For simplicity, lets assume that the weight matrices are identity matrices, and  all biases are zero.
htminus1 = np.array( [0, 0.5, 0.1, 0.2, 0.6] )
xt = np.array( [-0.1, 0.3, 0.1, -0.25, 0.1] )

I understand that forget state is sigmoid of htminus1 and xt
So, is it?
ft = 1 / ( 1 + np.exp( -( htminus1 + xt ) ) )

>> ft = array([0.47502081, 0.68997448, 0.549834  , 0.4875026 , 0.66818777])

I am referring to this link to implement of one iteration of one block LSTM. The link says that ft should be 0 or 1. Am I missing something here?
How do I get the forget gate layer as per schema given in the below mentioned picture? An example will be illustrative for me.

Along the same lines, how do I get the input gate layer, it and vector of new candidate values, \tilde{C}_t as per the following picture?

Finally, how do I get the new hidden state ht as per the scheme given in the following picture?
A simple, example will be helpful for me in understanding. Thanks in advance.

This is an image to better understand lstm... At $f_t$, we are taking the sigmoid of a weight matrix * the input at the current timestep + another weight matrix * $h_{t-1}$
Code Sample for $f_t$:
import numpy as np
import math

def sigmoid(values):
    sigmoid_applied = []
    for value in values:
        result = 1 / (1 + math.pow(math.e, -value))
        sigmoid_applied.append(result)
    return np.array(sigmoid_applied)

w1 = np.random.uniform(0, 1, size=[hidden_vector_len, input_len])
w2 = np.random.uniform(0, 1, size=[hidden_vector_len, hidden_vector_len])

f_t = sigmoid(np.dot(w1, input) + np.dot(w2, prev_hidden_state)) # Its matrix multiplication and not just simple multiplication

Note - There is also a bias term which I haven't included here for simplicity
If you understood $f_t$, you can do the same for other states also.
If you feel I am wrong anywhere in this post, then please do consider adding a comment


How do you efficiently choose the hyper-parameters of a neural network? How do you efficiently choose the hyper-parameters of a neural network (e.g. the learning rate, number of layer, weights, etc.)?

In deep learning era, there are two possible choices. Caveat approach and Panda approach.

Caviar Approach

In this approach, it is supposed that you have a very powerful cluster system that enables you to run different models simultaneously on different nodes. In this manner, you construct a d-dimensional space which each corresponds to a special hyperparameter. Then you can have a grid approach to partition the space and for each intersection of the grids, you can have a possible initialisation although there are some better ways in order to find better initialisations.

Panda Approach

This approach is widely used among students due to using desktop computers. In this approach, you usually try to find an initialisation of hyperparameters based on your experience or the other architectures' which are available and you try to refine them step by step by using cross-validation.

To answer your question, the efficiency depends on your computing power. For more details, you can take a look at the contents of the third week.


Prove that in such cases, it is possible to ﬁnd an ERM hypothesis for $H_n$ in the unrealizable case in time $O(mnm^{O(n)})$ Let $H_1$ , $H_2$ ,... be a sequence of hypothesis classes for binary classiﬁcation.

Assume that there is a learning algorithm that implements the ERM rule in the realizable case such that the output hypothesis of the algorithm for each class $H_n$ only depends on $O(n)$ examples out of the training set. Furthermore, assume that such a hypothesis can be calculated given these $O(n)$ examples in time $O(n)$, and that the empirical risk of each such hypothesis can be evaluated in time $O(mn)$.

For example, if $H_n$ is the class of axis aligned rectangles in $R^n$ , we saw that it is possible to ﬁnd an ERM hypothesis in the realizable case that is deﬁned by at most $2n$ examples.

Prove that in such cases, it is possible to ﬁnd an ERM hypothesis for $H_n$ in the unrealizable case in time $O(mnm^{O(n)})$.

Here's a rough proof sketch that might be clearer, but is probably less precise, than the solution you already have.

We have a sequence of hypotheses classes, but the problem actually only asks us to consider a single class. A hypothesis class is a set of possible models for classification that differ only in a set of parameter values.
We are told that the algorithm for selecting a hypothesis from $H_n$ uses only $O(n)$ samples from the training set, and that using those samples, it takes the algorithm $O(n)$ to find the hypothesis that minimizes the empirical risk for the class, provided that the ERM minimizing hypothesis has an empirical risk of 0 (i.e. it is realizable).
Further, if we pick some hypothesis at random from the class, we are told that the cost of determining its exact empirical risk is $O(nm)$, where $m$ is the total size of the training set, and $n$ is just a parameter associated with this particular hypothesis class $H_n$.
Observer that, in the unrealizable case, we cannot use the algorithm to find the risk minimizing hypothesis (which, by 2, runs in O(n)). No other algorithm is mentioned, but we are also told (in 2) that the algorithm can construct a hypothesis from $O(n)$ datapoints in $O(n)$ time, even if no feasible hypothesis can be found for those $O(n)$ points.
Suppose we wanted to find the hypothesis with minimal risk via brute force. We could compute every subset of the training dataset that is of the size $O(n)$ that the algorithm may be able to use to construct a hypothesis in $O(n)$ time. There are $O(m^{O(n)})$ such subsets (e.g. $m*(m-1)/2$ possible pairs, $m*(m-1)*(m-2)/6$ possible triplets, etc.).
Since there are $O(m^{O(n)})$ possible subsets of size $O(n)$, and our algorithm can compute a candidate hypothesis in $O(n)$ time, it costs us $O(nm^{O(n)})$ to find all possible hypotheses that our algorithm can produce for this dataset.
Since it costs us $O(nm)$ time to compute the emperical risk of a hypothesis generated by this algorithm, we can minimize risk by computing risk for each of the $O(m^{O(n)})$ hypotheses we made in step 6, and then selecting the one with lowest empirical risk. This costs us $O(mnm^{O(n)})$ time total. Our total runtime is then the sum of the cost to generate the models in step 6, and the cost to evaluate them here in step 7: $O(mnm^{O(n)})+O(nm^{O(n)})$.
Since $O(mnm^{O(n)})+O(nm^{O(n)}) \in O(mnm^{O(n)})$, we have proven the desired bound on runtime.


Can I perform multiclass classification when the number of features is less than the number of targets? Is it possible to perform multiclass classification on data where the number of features is less than the number of target variables? Do you have any suggestions on how to address a problem where I have 2000 target variables?

Of course. It only depends if those features are informative enough for the task at hand. In order to better understand the phenomenon, you can imagine 2 features displayed as points in a 2D plane. The number of possible target classes goes up to the number of clusters you can find in that plane.

About the suggestion, I can only recommend the utilisation of a non-linear classifier.


What is the relationship between fuzzy logic and objective bayesian probability? I understand fuzzy logic is a variant of formal logic where, instead of just 0 or 1, a given sentence may have a truth value in the [0..1] interval. Also, I understand that logical probability (objective bayesian) understands probability as an extension of logic, where uncertainity is taken into account. To me they sound rather similar (they both extend formal logic by modelling truth as a continuos interval between 0 and 1).

My question is, what is the relationship between these two concepts?. What is the difference, and what are the differences in AI approaches based upon these two formal systems?

This was a somewhat hotly debated question in the 1980s. The debate was more-or-less ended with papers like Cheeseman's In Defense of Probability.

The short answer is that Fuzzy Logic does not just assign a continuous value to sentences, what it does is assign degrees of membership in different fuzzy sets. These degrees of membership range between 0 and 1.

In contrast, probability says: Among the set of values this variable could take on, what fraction of them are in a certain set? This fraction also ranges between 0 and 1.

This might seem like a semantic distinction, but it has deep implications if you try to use these systems to reason about uncertainty.

For example, consider the question "Will it rain tomorrow?". A probabilistic approach would try to determine the fraction of days like tomorrow that have had rain. A fuzzy logic approach will try to determine whether tomorrow is like a rainy day. The distinction becomes obvious if we then ask whether it will not rain tomorrow. The probabilistic approach will try to find the fraction of days like tomorrow where it did not rain. The chance of it raining or not raining will sum to 1. The fuzzy logic approach will try to determine whether tomorrow is like a non-rainy day. Note that tomorrow may be both like a stereotypical rainy day and not like one. There is no firm requirement that these sets be disjoint. This reflects Cheeseman's critique of Fuzzy Sets: they implicitly reject the additive axiom of probability theory, which (as denoted by the name axiom) is something that is fairly unreasonable to reject.

While there are various approaches to make memberships in fuzzy sets more probabilistic in nature, that's the root distinction.

I think the dominant modern view is that fuzzy sets are a great tool when you need to reason about membership in a fuzzy concept. Whether rice is "done" or not isn't a question of something being true or false. It's a degree of membership in the set "done". On the other hand, whether the object in front of a self-driving car is a person or not is either true or false. It is not necessarily a good idea to reason about this in terms of whether the object is partially in the person set or plastic bag set.


Inception neural network input layer confusion According to the original paper on page 4, 224x224x3 image is reduced to 112x112x64 using a filter 7x7 and stride 2 after convolution.

nxn = 224x224

fxf = 7x7

stride = s = 2

padding = p = 0

Output of convolution is (((n+2p-f)/s)+1) according to which

(n+2p-f)=(224+0-7)=217,
divide by stride = 217/2=108.5 (taking the lower value), adding 118+1=119.

How do we get a output image of 112 now?

The padding is not size zero* in the inception CNN layers. In fact it is deliberately chosen to pad so that the convolution by itself would produce an image the same size as the original. I.e. $p=(f−1)/2$, in some libraries this is called "same" padding.

So, $p=3$

The stride is not 2. It is $s=1$ for the convolution. The Inception CNN does not use strided convolutions. Instead the stride of 2 is associated with a later max-pooling layer.

Therefore, using $(((n+2p-f)/s)+1)$ with the correct values $(((224 + 6 - 7)/1)+1 = 224$

Then apply max-pooling, with stride 2. $224/2 = 112$.

* Not to be confused with "zero padding" which means pad using $0$ as the value to insert into the new area. So you can have "zero padding with $p=3$"


What is a "codon" in grammatical evolution? The term codon is used in the context of grammatical evolution (GE), sometimes, without being explicitly defined. For example, it is used in this paper, which introduces and describes PonyGE 2, a Python library for GE, but it's not clearly defined.  So, what is a codon?

Grammatical evolution
To understand what a codon is, we need to understand what GE is, so let me first provide a brief description of this approach.
Grammatical evolution (GE) is an approach to genetic programming where the genotypes are binary (or integer) arrays, which are mapped to the phenotypes (i.e. the actual solutions, which can be represented as trees, which, in turn, represent programs or functions), using a grammar (for example, expressed in Backus-Naur form). So, the genotypes (i.e. what is mutated, combined, or searched) and the phenotypes (the actual solutions, which are programs) are different in GE, and the genotype needs to be mapped to the phenotype to get the actual solution (or program), but this is not the case in all GP approaches (for example, in tree-based GP, the genotype and the phenotype are the same, i.e. trees, which represent functions).
Codons
In GE, a codon is a subsequence of $m$ bits of the genotype (assuming that genotypes are binary arrays). For example, let's say that we have only two symbols in our grammar, i.e. a (a number) and b (another number). In this case, we only need 2 bits to differentiate the two. If we had 3 symbols,  a, b and + (the addition operator), we would need at least a sequence of 2 bits to define each symbol. So, in this case, we could have the following mapping

a is represented by 00 (or the integer 0),
b is represented by 01 (or 1), and
+ is represented by 10 (or 2)

The operation a+b could then be represented by the binary sequence 001001 (or the integer sequence 021). The 2-bit subsequences 00, 01 and 10 (or their integer counterparts) are the codons.
What do we need codons for?
In GE, codons are used to index the specific choice of a production rule. To understand this, let's define a simple grammar, which is composed of

a set of non-terminals (e.g. functions) $N = \{ \langle \text{expr} \rangle, \langle \text{op} \rangle, \langle \text{operand} \rangle, \langle \text{var} \rangle \}$
,
a set of terminals (e.g. specific numbers or letters) $\mathrm{T}=\{1,2,3,4,+,-, /, *, \mathrm{x}, \mathrm{y}\}$,
a set of production rules $P$, and
an initial production rule $S = \langle \text{expr} \rangle $.

In this case, the set of production rules $P$ is defined as follows
\begin{align}
\langle \text{expr} \rangle & ::=
\langle \text{expr} \rangle \langle \text{op} \rangle \langle \text{expr} \rangle \; | \;  \langle \text{operand} \rangle \\
\langle \text{op} \rangle & ::= + \;  | \;  - \;  | \; * \; | \; / \\
\langle \text{operand} \rangle & ::= 1 \;  | \; 2 \; | \;  3 \; | \;  4 \;  | \;  \langle \text{var} \rangle \\
\langle \text{var} \rangle & ::= \mathrm{x} \; | \;  \mathrm{y}
\end{align}
So, there are four productions rules. To be clear, $\langle \text{var} \rangle  ::= \mathrm{x} \; | \;  \mathrm{y}$ is a production rule. The symbol $|$ means "or", so the left-hand side of each production is a non-terminal (and note that all non-terminals are denoted with angle brackets $\langle \cdot \rangle$), which is defined as (or can be replaced with) one of the right-hand choices, which can be a non-terminal or terminal. The first choice of each production rule is at index $0$. The second choice at index $1$, and so on. So, for example, in the case of the production $\langle \text{var} \rangle  ::= \mathrm{x} \; | \;  \mathrm{y}$, $\mathrm{x}$ is the choice at index $0$ and $\mathrm{y}$ is the choice at index $1$.
The codons are the indices that we use to select the production rule's choice while transforming (or mapping) the genotype into a phenotype (an actual program).
So, we start with the first production, in this case, $S = \langle \text{expr} \rangle$. If it's a non-terminal, then we replace it with one of its right-hand side choices. In this case, there are two choices

$\langle \text{expr} \rangle \langle \text{op} \rangle \langle \text{expr} \rangle$ (choice at index $0$)
$\langle \text{operand} \rangle$ (choice at index $1$)

If our genotype (integer representation) is, for example, $01$ (note that this is a sequence of integers), we would replace $\langle \text{expr} \rangle$ with $\langle \text{expr} \rangle \langle \text{op} \rangle \langle \text{expr} \rangle$, then we would replace the first $\langle \text{expr} \rangle$ in $\langle \text{expr} \rangle \langle \text{op} \rangle \langle \text{expr} \rangle$ with $\langle \text{operand} \rangle$, so we would get $\langle \text{operand} \rangle \langle \text{op} \rangle \langle \text{expr} \rangle$, and so on and so forth, until we get an expression that only contains terminals, or the genotype is terminated. There can be other ways of mapping the genotype to the phenotype, but this is the purpose of codons.
Codons in biology
The term codon has its origins in biology: a subsequence of 3 nucleotides is known as a codon, which is mapped to an amino acid in order to produce the proteins. The set of all mappings from codons to amino acids is known as genetic code. Take a look at this article for a gentle introduction to the subject. So, in GE, codons also have a similar role to the role of codons in biology, i.e. they are used to build the actual phenotypes (in biology, the phenotypes would be the proteins or, ultimately, the organism).
Notes
Codons do not have to be 2-bit subsequences, but they can be $m$-bit subsequences, for some arbitrary $m$. The term codon is similar to the term gene, which is also often used to refer to specific subsequences of the genotype (for example, in genetic algorithms), although they may not be synonymous (at least in biology, genes are made up of sequences of codons, so they are not synonymous). Moreover, the binary $m$-bit codons can first be mapped to integers, so codons can also just be integers, as e.g. used here or illustrated in figure 2.2 of this chapter.
Further reading
You can find more info about codons in the book Genetic Programming: An Introduction by Wolfgang Banzhaf et al., specifically sections 9.2.2. (p. 255), 9.2.3. (an example) and 2.3 (p. 39), or in chapter 2 of the book Foundations in Grammatical Evolution by Dempsey et al.


Can a computer conclude following philosophical concepts from a story? Say you have to enter a story to a computer. Now the computer has to identify the philosophical concept on which the story is based, say:

Was it a "self-fulfilling prophecy"?
Was it an example of "Deadlock" or "Pinocchio paradox situation"?
Was it an example of how rumours magnify? or something similar to a chain reaction process?
Was it an example of "cognitive dissonance" of a person?
Was it a story about "altruism"?
Was it a story about a "misunderstanding" when a person did something "innovative" but it accidentally was innovated earlier so the person was "falsely accused" of "plagiarising"?

et cetera,

Given that the story is not only a heavy rephrase of the pre-existing story; not only character names and identities are totally changed, but the context completely changed,  the exact tasks they were doing are changed.

Can computers identify such "concepts" from stories? if yes, then what mechanism it uses?

No. This is currently out of the scope for any language processing system. It requires a general understanding of abstract concepts which is not possible for machines at present.

In order to recognise a self-fulfilling prophecy, you first need to identify that something is a prophecy. So it needs to be something that expresses a possible future state, for which you need to identify what possible future states are; and then you need to see whether it is self-fulfilling. Conceptually this is far too complex to do.

You might get away for some of these with formal criteria (eg use of future tense for something describing a future state/event), but this is far too imprecise.

"Altruism" requires knowledge about typical expected behaviour; you would need to be able to identify motives behind people's actions, and then decide whether it was altruistic or not. This is just too complex for now (and the foreseeable future).


Does a differential evolution algorithm mutate its population during a generation? I'm implementing a differential evolution algorithm and when it comes to evolving a population, the page I am referencing is vague on how the new population is generated.
https://en.wikipedia.org/wiki/Differential_evolution#Algorithm
The algorithm looks like the population is mutated during evolution.
# x, a, b, c are agents in the population
# subscriptable to find position in specific dimension
# pop is filled with Agent objects
for i, x in enumerate(pop):
  [a, b, c] = random.sample(pop[:i]+pop[i+1:], 3)
  ri = random.randint(0, len(x))
  new_position = []
  for j in range(len(x)):
    if random.uniform(0, 1) < CR or j == ri:
      new_pos.append(a[j] + (F * (b[j] - c[j])))
    else:
      new_pos.append(x[j])
  # Agent() class constructor for agent, takes position as arg
  new_agent = Agent(new_pos)
  if fitness(new_agent) <= fitness(x):
    pop[i] = new_agent # replace x with new_agent

But I wonder if instead it means a new population is made and then populated iteratively:
new_pop = []
for i, x in enumerate(pop):
  [a, b, c] = random.sample(pop[:i]+pop[i+1:], 3)
  ri = random.randint(0, len(x))
  new_position = []
  for j in range(len(x)):
    if random.uniform(0, 1) < CR or j == ri:
      new_pos.append(a[j] + (F * (b[j] - c[j])))
    else:
      new_pos.append(x[j])
  new_agent = Agent(new_pos)
  if fitness(new_agent) <= fitness(x):
    new_pop.append(new_agent)
  else:
    new_pop.append(x)
pop = new_pop

Note new_pop is made, and filled with agents as the for loop continues.
The first allows previously evolved agents to be used again in the same generation; in other words, the population is changed during the evolution. The second doesn't allow updated agents to be re-used, and only at the end is the original population changed.
Which is it?

Quoting the original paper:

For each target vector $x_{i,G}$ ,a mutant vector is generated according to
$$ v_{i,G+1} = x_{r_1,G} + F\left(x_{r_2,G} + x_{r_3,G}\right)$$

And later

To decide whether or not it should become a member of generation $G + 1$, the trial vector $v_{i,G+1}$ is compared to the target vector $x_{i,G}$ using the greedy criterion.

I'd say it is pretty unambiguous that author's intent was to split evolving agents' populations into "generations" indexed by $G$. A new generation $G+1$ is created by applying the DE algorithm to the previous generation $G$. No agents are changed within the generation $G$.
So, it looks like the answer to your question is "no" - the population of the current generation $G$ is not mutated. Your second approach is the correct one.