How to "forward" updated NN model to a transferred model? I've trained a robot to walk in a straight line for as long as it can (using TD3), and now I'm using that pre-trained model for two new models with separate purposes: 1. Walk to a specific point and halt (adding target position to the NN inputs); 2. Walk straight at a specified velocity (adding a target velocity to NN inputs).
Now let's say I retrain the original model again to walk properly after changing, say, the mass of the robot. How can I approach "forwarding" this update to the two transfer-learned models? The purpose of this is to minimize re-training time for all future models transfer-learned from the original.
(What strikes me as particularly challenging is the fact that the input layer of the transfer-learned models have additional features, so this may re-wire the majority of the NN, making a "forwarded update" completely incompatible...)

I think there is no simple way to transfer knowledge changes between different models.
If you take your initial model and create a new version of it which you use to learn some other task (like "Walk to a specific location"), then the values copied from the first (original) model change in the second model.
From that moment on, training the former model on another task will have different effects on its weights than continuing the training of the second model, whose parameter have been changed already.
Consider, for example, that you had changed the mass of the robot and trained the initial model on that new task already. Then, if you took all the re-trained parameters from the first model and implanted them into the second model (trained on walking to a certain location), then you would essentially overwrite the additional knowledge the second model had gained already during its initial transfer-learning-process (not even taking into consideration any additional parameters appended to the list of parameters in the second model).
So, you will have to re-train all three models (the original one and the two transfer-learning models) if you change the mass of the robot.
Edit:
There might be an option to apply the same knowledge changes to another model architecture if you refrain from pure transfer learning. This can be achieved with a more modularized model architecture.
Consider that you train your first model on walking straight head. Let's call this model $m_{walk}$.
Then you intend to recycle $m_{walk}$ for another task, like walking straight ahead to a given location. Such a model architecture could be realized in two ways:

You apply real transfer learning, retraining a copy of $m_{walk}$ to walk straight ahead until it reaches a certain location
You take $m_{walk}$, don't change it, but add a second model (let's call it $m_{navigator}$), which is trained on predicting $go$ vs. $stop$.

In the second case, your overall model architecture (let's call it $m_{go\_to}$) consists of both one model used for walking (i.e. $m_{walk}$) and one model architecture which is used for predicting $go$ vs. $stop$ (i.e. $m_{navigator}$).
The idea then is that the robot executes the actions suggested by $m_{walk}$ until $m_{navigator}$ suggests stopping, upon which prediction the suggestions by $m_{walk}$ will be ignored.
Then, whenever you retrain your model $m_{walk}$ (e.g. because the mass of some robot changes), you can simply apply the changes to $m_{go\_to}$ by replacing $m_{walk}$ by a new version, leaving the rest of $m_{go\_to}$ intact.
If you generalize $m_{walk}$ to not only walk straight ahead, but also to take turns etc. and you generalize $m_{navigator}$ to predict going $left$, $right$, $straight\ ahead$, $back$, or $stop$ (being predicted when a certain destination has been reached), you can generalize $m_{go\_to}$ to walk where-ever you want it to go.


Why is my fine-tuned YOLO model detecting other objects as a human? I am new to deep learning and computer vision. I have a problem where I use the YOLO to detect objects.
For my problem, I just want to recognize 1 human only. So, I changed the final YOLO's layer (which contained 80 neurons) to only 1 neuron, and do the training process with transfer learning techniques. Of course, I do not use the final layer's weights, and these weights are randomly initialized for my problem. I feed only the human data to the model.
However, I realize that after longer training, the model becomes worse. It starts to recognize other objects as a human.
Should I also feed non-human data to the model?

So you have a network pretrained on 80 classes. I also assume that one of these classes are human (or else this is just not the way to go*) I suspect that the final layer contains 80 labels, correct? Then you then 'rescale' this layer to 1 label and then train on some data you possess? Then you're basically trying to teach the network that it shouldn't care about the 79 other classes, which is just nonsense I think.

What you could do, and I do not recommend this, but if you feel like you have to use this exact network, you just keep the 80 outputs and only look at the label correspondning to the human.

You shouldn't do this because the network is WAY bigger than it needs to be to only classify human/non human, which will make it slower than it needs to be.

What you rather want to do is either to train your own network (If you have lots of training data, I suspect this wouldn't be be the hardest thing to train) or obtain a CNN that is pretrained on human classification.

(*I've heard rumours that you can do pretty well on retraining a class on a pretrained network. I just don't know if the rumours are true or how to go about it.)


What will change when workstations will have ARM Machine Learning Processors onboard? lately we read that many manufacturers are forcing ARM architectures to be used on future workstations. One of ARM's recent announcements is a machine learning processor. What will change in terms of computing performance if ARM architectures become new standard, and these kinds of ML-focused chips are found in most devices?

It is not clear to me that much will change.

ARM makes many devices, mostly designed to consume as little power as possible. My guess is that most workstations will not contain ARM's ML processor, even if they contain an ARM CPU.
ARM's ML processor can do machine learning. It is specifically optimized for training convolutional deep neural networks, and it is a bit faster that most existing non-purpose built chips, especially for its price. In practice, chips like this one should make ML a bit cheaper and faster, but it's not a breakthrough product. This just looks like the next iteration of Moore's law.


Try lowering the learning rate.

Such a loss curve can be indicative of a high learning rate. Due to a high learning rate the algorithm can take large steps in the direction of the gradient and miss the local minima. Then it will try to come back to the minima in the next step and overshoot it again.

You may also try switching to a momentum-based GD algorithm. Such a training loss curve can be indicative of a loss contour like in this example, for which momentum-based GD methods are helpful.

I noticed that you have a very small training set. You may have better luck with a larger training data (~1000 examples) or using a pre-trained Conv network as a starting point.


Why is dot product attention faster than additive attention? In section 3.2.1 of Attention Is All You Need the claim is made that:

  Dot-product attention is identical to our algorithm, except for the scaling factor of $\frac{1}{\sqrt{d_k}}$. Additive attention computes the compatibility function using a feed-forward network with a single hidden layer. While the two are similar in theoretical complexity, dot-product attention is much faster and more space-efficient in practice, since it can be implemented using highly optimized matrix multiplication code.

It does not make sense why dot product attention would be faster. Additive attention is nearly identical computation wise; the main difference is $Q + K$ instead of $Q K^T$ in dot product attention. $Q K^T$ requires at least as many addition operations as $Q + K$, so how can it possibly be faster?

The additive attention method that the researchers are comparing to corresponds to a neural network with 3 layers (it is not actually straight addition). Computing this will involve one multiplication of the input vector by a matrix, then by another matrix, and then the computation of something like a softmax. Smart implementation of a dot-product will not break out the whole matrix multiplication algorithm for it, and it will basically be a tight, easily parallelized loop.


Genetic Algorithm vs Particle Swarm Optimization Which one gives better optimization results? Genetic Algorithm or Particle Swarm Optimization? Can I use them for online tuning problems? Thanks in advance!

These kind of questions cannot be answered without looking at a particular project. Each algorithm has its particular strengths and weaknesses; and trade-offs in terms of use of resources (processing power and/or storage space, for example). If there was an objective answer, then the worse algorithm would surely fall in disuse.

It also depends what you mean by "better". Faster? Better score according to some evaluation measure? More robust (ie works with many diverse data sets)?

I would recommend looking at both algorithms in more detail, and trying to understand how they work. Then you should be able to find out which best fits your problem.

However, one problem with Particle Swarm Optimisation is that it is not well understood, so you might have to resort to trial-and-error.


Is there any resource that describes in detail a naive example-based machine translation algorithm? I'm looking to develop a machine translation tool for a constructed language. I think that the example-based approach is the most suitable because the said language is very regular and I can have a sufficient amount of parallel translations.
I already know the overall idea behind the example-based machine translation (EBMT) approach, but I can't find any resource that describes a naive EBMT algorithm (or model) that would allow me to easily implement it.
So, I'm looking for either:

a detailed description,
pseudocode or
a sufficiently clear open-source project (maybe a GitHub one)

of a naive EBMT algorithm. So, I'm not looking for a software library that implements this, but I'm looking for a resource that explains/describes in detail a naive/simple EBMT algorithm, so that I am able to implement it.
Note that there are probably dozens of variations of EBMT algorithms. I'm only looking for the most naive/simple one.
I have already looked at the project Phrase-based Memory-based Machine Translator, but, unfortunately, it is not purely based on examples but also statistical, i.e. it needs an alignment file generated by, for example, Giza++ or Moses.

I have not found any simple implementation of a naive EBMT system, but I found some articles, papers and books that may be helpful (although I haven't read them, apart from the first and last one), so I will list them below.

The web article Example-based machine translation provides a decent high-level explanation of example-based machine translation.

The paper Example-Based Machine Translation: A New Paradigm (2002) by Chunyu Kit et al. also seems to provide a detailed description of the EBMT approach, so this paper should provide you with details you need to implement an EBMT system.

The paper A framework of a mechanical translation between Japanese and English by analogy principle (1984) by Makoto Nagao introduced the example-based machine translation approach, so it will be at least historically relevant.

Additionally, the paper Example-Based Machine Translation of the Basque Language and the book Recent Advances in Example-Based Machine Translation (2003), which is not apparently freely available online, could also be useful.

Finally, the article Machine Translation. From the Cold War to Deep Learning gives a nice high-level overview of the main machine translation approaches, so that you can understand the differences between EBMT and other approaches (especially, in case you are not able to distinguish between EBMT and other MT, e.g. those that use a parallel corpus, such as the supervised statistical machine translation approaches).


Define rules for the predicate Just started reading a book about AI. There is a very basic exercise but I can't figure it out, so here we go. The book is  Simply Logical: Intelligent Reasoning by Example

The exercise is in the page 19.

  Two stations are ‘not too far’ if they are on the same or a different
  line, with at most one station in between. Define rules for the
  predicate not_too_far.

The only rules I've seen are nearby and connected and don't know how to use this. What I've done so far is this:

  not_too_far(X,Y) :- nearby(X,Y)

Your intuition is good. Because "nearby" is only defined with "connected", there could only be 1 station between them. However, it says that the stations are "not_too_far" if at most one station is between them. What about if no stations are between them? If 2 stations are "connected" they should be "not_too_far" as well.

So it should be:

not_too_far(X,Y) :- connected(X,Y) ; nearby(X,Y).

Where ; denotes OR.


Detect observations under certain conditions I have a customer purchasing dataset and the data set is from a retailer having an online store and offline stores. So, customers have two options in their shopping channel, online or offline. In an online shopping, there is a shipping fee however if a basket size is larger than $50 there is no shipping fee.

I found pieces of evidence that customers are trying to add some of items to make their basket size larger than $50 when their baskets are near and a little bit below the $50, because their shipping fee can be waived by doing that.

In this situation, I am trying to identify and characterize items that were purchased only because of the shipping threshold by using a machine learning algorithm.

If there is no shipping threshold, $50, the customers would not purchase the items, but they purchased some items to make their basket size larger than $50. I have not observed those kinds of items (added items because of the shipping threshold).

Is there any machine learning algorithm that I can identify those kinds of items?

I think I need to use some of unsupervised machine learning algorithm.

Another challenging part is that each customer has different characteristics so I probably need to consider it as well. How can I detect those kinds of items??

Since my comment would not fit, I'll answer the question.

I think this problem will be even more better solved if you have the web data as well. Say after adding the required necessity in the cart the customer will check the total amount. And if it is below 50$, the person will add some more items. So checking this can give you a better clue.
Another data you must have for better guessing is the order in which the person has added items to his cart. This will also provide you an important clue about the cheap items the customer is adding to cart to try to cross the minimum threshold.
You will also have to segregate cheap items from pricey ones for a customer. This data with the above 2 approaches is a definite giveaway.
This might be a personal opinion, but whenever I need to cross the shipping fee threshold I usually add items which I have already bought the previous time to escape shipping fee. You will need more data to see if this is true in most cases.
And if you don't have any of the above data, the pure Machine Learning approach would be to pick an item which is costly and somewhat popular and an item which will string together people with likewise interests. Say, a great book on ML is this item.
You find out all the customers who bought this book. Check all the similarities between all the other things bought by the customer, like say one customer adds a book on AI after this while another buys a book on python. So these are related by computer science. Go on until you find all the things that are generally dissimilar between all the customers, since now they are buying to cross the threshold and each will buy according to his own requirements without a common interest in mind. There you have all the data you need.
Or you can use the converse of this approach, find a non-significant thing like sugar. String together all the customers who bought this item. Check their interests to see if it has anything to do with sugar. If not send them to one group. Now in this group, match the interests of the customer on other items, if dissimilar you get a good idea that this item was bought to cross threshold.

I understand this is a kind of opinionated answer. But I think these are trade secrets not revealed by companies. so you have to figure out algorithms yourself. Also, simple machine learning won't work, lots of logical programming is also required. And I also think you need to have a good understanding of human psychology works. You can interview your friends and family about what they would do and get a general intuition. Hope this helps!


Could humans hurt a conscious or sentient AI? If a conscious AI is possible, would it also be possible for someone who knows what they are doing to torture (or hurt) the AI? Could this be avoided? How?

This question deals with computer based AI, not robots, which are as conscious as people (this is an assumption of the question). The question wonders how a crime as hard to trace as illegal downloads, but far worse ethically, could be prevented. Note that despite most people being nice and empathising with the robots, there are always the bad people, and so relying on general conscience will not work.

The article Children Beating Up Robot Inspires New Escape Maneuver System is based on two research papers about an experiment in a Japanese mall that led to unsupervised children attacking robots. The research paper you're interested in is Escaping from Children’s Abuse of Social Robots.

In that research paper, researchers were able to program the robots to follow a planning simulation to reduce the probability of abuse by children. If it detects children, the robot is programmed to retreat into a crowd of adults (who can then discipline the children if needed). This happened because the researchers saw that it was only children who were beating up the robots in the mall in question.

They discuss trying out other options though:

  In this work the robot’s strategy to prevent abuse was to “escape”, i.e. move to a location where it is less likely abuse will occur. One could ask why the robot cannot overcome the abuse. In our preliminary trials we have tried several approaches, but we found that it is very difficult for the robot to persuade children not to abuse it. For example, we changed the robot’s wordings in many ways, using strong words, emotional or polite expressions, but none of them were successful. One partially successful strategy was the robot ‘physically’ pushing children. When its way was blocked, it would just try to keep going and behave as if it will collide into children and force its way through (under careful monitoring from a human operator). We observed that children at first accepted the robot’s requests and obeyed them; but, very soon they learned that they are stronger than the robot so they can win if they push, and also that they can stop it by pushing the bumper switch (attached on the robot for safety). After realizing that, they just continued with the abusive behavior. Obviously having a stronger robot would present a problem for safety and social acceptance so dealing with such abusive situations remains difficult.

But let's interrogate your question further:

  If conscious AI is possible and is wide spread, wouldn't it be easy for someone who knows what they are doing to torture AI?

Why would you consider such torture to be wrong? After all, one could argue that the machine won't really 'experience' pain if you torture it...so it should be morally okay to torture the machine then. It may be respond as if it is in pain, but it's dubious whether the ability to simulate an emotional state such as "being in pain" is equivalent to actually being in that emotional state. See the question Is the simulation of emotional states equivalent to actually experiencing emotion? for more discussion on this topic.

You can make such an argument, but it won't really work on an emotional level because most humans would feel empathy towards the machine. It may be hard to justify it logically (and it may be based on humans' tendencies to engage in anthropomorphism), but we feel this empathy. It's this empathy that caused you to ask this question in the first place, caused researchers to figure out how to protect a robot from being beaten up, enabled police officers to arrest a drunken Japanese man for beating up a SoftBank robot, and made many humans upset over the destruction of hitchBOT. And that's how AI abuse would be avoided - human empathy. If most humans care about the welfare of machines, they'll make it a priority to stop those few humans who are able and willing to abuse the machines.

EDIT: The OP has edited his question to clarify that he is talking about software, and not about robots. For robots, you can rely on anthropomorphism to produce some level of sympathy, but it's hard to sympathize with raw lines of code.

You're not going to stop abuse of algorithms. Put it frankly, since the algorithms aren't like us, we aren't going to extend the same sort of empathy that we would to robots. Even chatbots are kinda iffy. If you could get people to sympathize with lines of code though (possibly by making a convincing simulation of emotion and sapience), then the above answer applies - humans anthropomorphize the machine and will come up with countermeasures. We aren't that level yet, so "stopping AI abuse" will be a low priority.

Still, some failsafes could be programmed in to limit the damage of abuse, as detailed in this thread on chatbot abuse - making the bot respond in a boring manner to make the abuser feel bored and move onto the next target, responding back to the abuser in a "battle of wits", or even just blocking the abusers from using the service.

These failsafes are cold comfort to those that want to prevent abuse, not respond to it.

Also...an abuser can happily learn how to program an AI to then abuse to his/her heart's content. Nothing can be done to stop that, and any possible measures to stop said abuse (such as monitoring every human being to make sure they don't program an AI to abuse) will probably cause more damage than it'd solve.


Why does the classifier network in RPN output two scores? The region proposal network (RPN) in Faster-RCNN models contains a classifier and a regressor network. Why does the classifier network output two scores (object and background) for each anchor instead of just a single probability? Aren't the two classes considered exclusive?

Source: Figure 3 of the original Faster-RCNN paper

I just want to provide this intuition

this NN consists of a 2 steps detection pipeline (the region proposal and regression + classification in parallel) exploring a certain range of scales and aspect ratios for proposals
as the proposed region is rectangular and the objects of interests have not a rectangular appearance strictly speaking, but a mostly rectangular one (with different ARs), considering the resulting detection bboxes areas some pixels will belong to a certain relevant object appearance (e.g. pixels of a pedestrian, car, …) while other pixels in that BBox won’t be related to that semantic
as this is not semantic segmentation we can now know what pixel is actually relevant, however the objectness and backgroundness can provide a rough measure of this in an aggregated way

Let’s assume at some point in your processing pipeline, you have 2 regions containing the same car but at different scales: so ideally both of them should be associated with good confidence to CarID but it’s clear one will contain much more background pixels than the other one and you'd like this to be represented as an additional measure


What class of problem is this? If I have a lot of input output pairs as training data

<float Xi, float Yi>

and I have a parametrized approximation function (I know the function algorithm, but not the values of the many many parameters it contains) which shall approximate the process by which the original data pairs were generated. The function takes two input values:

// c is a precomputed classifier for x and can have values from 0 to 255, so there can be up to 256 different classes
y = f(float x, int c)

the hidden parameters of the function are some big lookup tables (a lot of free parameters, but still much fewer than the amount of data points in the training data)

Now I want to fit all the hidden parameters that f contains AND compute for each Xi a ci, such that for the fitted function the error over all i of Yi - f(Xi, ci) is minimized

So, using some algorithm I want to fit the parameters of f and also classify the inputs Xi so that f(Xi, ci) aproximates Yi

How is this kind of problem called and what kind of algorithm is used to solve it?

I assume it's possible to initialize all hidden parameters as well as all ci with random values and then somehow use back propagation of the error to iteratively find  parameters and ci such that the function works well.

What I don't know is whether this is a well known class of problem and I just don't know the name of it, so I'm asking for pointers.

Or maybe in other words:
I have a function that has a certain layout (for performance reasons) which I want to use to approximate and interpolate my training data, I want to tune the parameters of this function such that it approximates the original data well. since the data points fall into some 'categories', I want to pre-classify the data-points to make it easier for the function to do its job. What kind of algorithm do I use to find the function's parameters and to pre-classify the input?

This problem is typically called parameter estimation or inverse modelling, and there are a variety of techniques to solve it.

If your free parameters are all continuous (i.e. none are discrete, such as integers), and your model function is differentiable, then you can turn the model into a computation graph in e.g. TensorFlow and use gradient descent methods with MSE loss, learning parameters on the data in much the same way as a neural networks. Most of the toolkits built for neural networks can do this, you just have to ignore the pre-packaged layer models and the utilities to manage them. You will need to take care with parameter initialisation, ideally you already have some rough starting values.

You can use other approaches, such as search methods too. Even when you have some discrete-valued parameters then you can use genetic algorithms for instance.

The best method to use will depend on details of your model, it is not possible to point out a generic "best". If you can use a gradient-based method directly, as you suggest in the question, then that might be the most efficient in terms of computation, provided you have some way to set reasonable initial parameters.


Is there some known pattern for selecting a batch of candidates for the next generation? I'm a beginner with a classic "racing car" sandbox and a homemade simple neural network.
My pattern:

Copy the "top car" (without mutation) to the next generation

If there are some cars still running (because simulation reached the 30s win condition), then copy a mutated version of them for the next generation.

Fill the rest of the pool with mutation of the "top car".

But this is just some dumb intuitive pattern I made on the fly while playing with my code. Perhaps I should copy the cars that are still running as-is instead of mutating them. Or, perhaps, some selection method I don't know about.
A new random track is generated at each new generation. a "top car" may be good on a track and crash immediately on the next track. I just feel that basing everything on the top car is wrong because of the track randomness.
Is there some known pattern for selecting a batch of candidates? (paper, google-fu keyword, interesting blog, etc.)
I don't know what to search for. I don't even know the name of my network or any vocabulary related to AI.

The most general descriptive frameworks covering what you are trying to do are:

Sequential decision making (article is a stub, but the term a good launching point to discover different wys of modelling and solving these kind of problems)

Optimal control

These put some context around your problem, and might give you some pointers. For instance, reinforcement learning is an alternative approach to the evolutionary system you are trying to build.
The specific AI system you appear to be building is a genetic algorithm, and more specific still you are attempting to find a neural network that is optimal at a task by searching for the best network using a system of population generation, selection and mutation which repeats.
There are lots of ways to set up a system like this, so your approach is not necessarily wrong. However, I think there are two key things that would improve what you have built so far:

Use a fitness function for selection. Score each car, perhaps by how far it got before crashing when the episode ends. To reduce luck factor on random courses, you could make this score the mean result from e.g. 3 different courses (it is not necessary, but may address your concern that selection is too random in your case). Select some fraction of top scoring cars, or look into other selection approaches - e.g. weighted selection based on fitness score or ranking.

Add "sex", more properly known as genome crossover between selected population members. Mutating individuals is limiting because it silos improvements to a single line of ancestry - if there are two good mutations found at random you rely on that single line finding both of them. Whilst crossover allows sharing of good mutations between lines, making it much more likely that two good mutations will end up in the same individual.

There is a framework called NEAT which covers the issues above plus has other features useful for evolving neural networks. It often does well at control scenarios like the one you are considering. You may want to look into it, if your focus is mainly on solving the control problem. However, it is relatively advanced from where you are, so if your current focus to learn by building from scratch you may get more initially from implementing fitness functions and crossover yourself.


What's the advantage of log_softmax over softmax? Previously I have learned that the softmax as the output layer coupled with the log-likelihood cost function (the same as the the nll_loss in pytorch) can solve the learning slowdown problem.

However, while I am learning the pytorch mnist tutorial, I'm confused that why the combination of the log_softmax as the output layer and the nll_loss(the negative log likelihood loss) as the loss function was used (L26 and L34).

I found that when log_softmax+nll_loss was used, the test accuracy was 99%, while when softmax+nll_loss was used, the test accuracy was 97%.

I'm confused that what's the advantage of log_softmax over softmax? How can we explain the performance gap between them? Is log_softmax+nll_loss always better than softmax+nll_loss?

The short answer is yes, log_softmax + nll_loss will work better.

I don’t know the implementation details under the hood in PyTorch, but see the screenshot below from the documentation:


Concrete examples of unintentional adversarial AI behaviour Are there any real-world examples of unintentional "bad" AI behaviour? I'm not looking for hypothetical arguments of malicious AI (AI in a box, paperclip maximizer), but for actual instances in history where some AI directly did something bad due to its direct action that was not intended behavior.

Interpretations of the meaning of "AI", "bad", and "unintentional" are left open due to obvious reasons. Be free with your interpretation, but use some common sense please.

Ex: Microsoft Tay became a racist not too long after being hooked up to the internet, thanks to internet trolls "teaching" her bad things.

I can't think of any other instances. So the following examples are just hypothetical scenarios to demonstrate what I mean.

Ex: A self-driving car drove off-track after being presented with an adversarial example, crashing into people.

Ex: Surgery bot goes haywire and accidentally kills someone.

Ex: Weaponized drone targets civilians against its design.

There is the case of the tesla accident where the car was in autopilot and crashed into a truck because it appears the vehicle mistook a lightly coloured truck for the sky, killing the driver: https://www.newscientist.com/article/2095740-tesla-driver-dies-in-first-fatal-autonomous-car-crash-in-us/

Having said that, it appears the car had been trying to tell the driver to pay attention: http://uk.reuters.com/article/us-tesla-crash-idUKKBN19A2XC


Interpretation of feature selection based on the model The description of feature selection based on a random forest uses trees without pruning.
Do I need to use tree pruning?
The thing is, if I don't cut the trees, the forest will retrain.

Below in the picture is the importance of features based on 500 trees without pruning.

With a depth of 3.

I always use the last four signs 27, 28, 29, 30.
And I try to add to them signs from 0 to 26 by means of cycles, going through possible combinations.
Empirically, I assume that the trait number 0, 26 is significant.
But, on both pictures it is not visible. Although the quality of classification with the addition of 0, 26 has improved.

random forest's feature importances are not reliable and you should probably avoid them. Instead you can use permutation_importance: https://scikit-learn.org/stable/auto_examples/inspection/plot_permutation_importance.html#sphx-glr-auto-examples-inspection-plot-permutation-importance-py


How do I normalise/un-normalise data when loading a model? I am following this TensorFlow JS tutorial where you load car data. The data looks like this:

[{x:100, y:20}, {x:80, y:33}]

X is the horsepower of a car, Y is the expected miles per gallon usage. After creating the model I save it locally using:

async function saveModel(){
    await model.save('downloads://cars-model');
}

Next, I load the model in a separate project, to make predictions without needing the original data.

NEW PROJECT

async function app(){
    let model = await tf.loadLayersModel('./cars-model.json');
    console.log("car model is loaded!");
}

I expect to be able to run predict here, on a single number (say, 120)

model.predict(tf.tensor2d([120], [1, 1]))

QUESTION

I think the number 120 needs to be normalised to a number between 0-1, just like the training data was. But how do I know the inputMin, inputMax, labelMin, labelMax values from the loaded model?

To un-normalise the prediction (in this case 0.6) I also need those original values.

How do I normalise/un-normalise data when loading a model?

original prediction code uses label and input values from the original data

function testModel(model, inputData, normalizationData) {
    const { inputMax, inputMin, labelMin, labelMax } = normalizationData;

    // Generate predictions for a uniform range of numbers between 0 and 1;
    // We un-normalize the data by doing the inverse of the min-max scaling
    // that we did earlier.
    const [xs, preds] = tf.tidy(() => {

        const xs = tf.linspace(0, 1, 100);
        const preds = model.predict(xs.reshape([100, 1]));

        const unNormXs = xs
            .mul(inputMax.sub(inputMin))
            .add(inputMin);

        const unNormPreds = preds
            .mul(labelMax.sub(labelMin))
            .add(labelMin);

        // Un-normalize the data
        return [unNormXs.dataSync(), unNormPreds.dataSync()];
    });

    const predictedPoints = Array.from(xs).map((val, i) => {
        return { x: val, y: preds[i] }
    });

}

How do I convert this to a number between 0-1 without having access to the original car data?

You save the normalisation parameters (typically an offset and a multiplier for each column), and consider that part of the model. Typically you do this when you originally scale training data.

When you want to re-use the model, as well as loading the neural network architecture and weights, you need to load the normalisation parameters in order to re-use those too and scale your inputs.

When tutorials present a self-contained neural network that loads training data, builds a model, then tests that model, all in the same process, then often this step is not shown. However, saving the normalisation data is important, and basically should be considered part of the model, even though it is not directly part of the neural network parameters or hyper-parameters.


Is there any programming language designed by deep learning? I know that AI can be used to design printed circuit boards (PCBs), so it can be used to solve complex tasks.
Is there any programming language designed by deep learning (or any other AI technique)?

There are certainly things like this.

I'd say a strong example is layered learning approaches, descended from Peter Stone's work.

A programming language is essentially a collection of useful shorthands for assembly-level instructions. Ultimately, everything you do in a programming language eventually gets executed in assembly. So making a programming language amounts to learning how to write short, reusable, assembly language programs that you can then use as building blocks to solve harder problems.

An example of this in action is Kelly & Heywood's approach to constructing 'Tangled Program Graphs' for reinforcement learning (IJCAI 2018). Here an evolutionary algorithm is used to learn short assembly programs, that can be combined into a graph to make more complex programs. This is similar to graphical programming languages like J.


Why does 'loss' change depending on the number of epochs chosen? I am using Keras to train different NN. I would like to know why if I increment the epochs in 1, the result until the new epoch is not the same. I am using shuffle=False, and np.random.seed(2017), and I have check that if I repeat with the same number of epochs, the result is the same, so not random initialization is working.

Here I attach the picture of the resulting training with 2 epochs:

And here I attach the picture of the resulting training with 3 epochs:

Also, I would like to know why the training time is not (3/2) and how is it possible that some of them have less accuracy with one more epoch.

Thanks a lot!

You are using two optimisers here: Stochastic Gradient Descent (SGD) and Adam (which is a more complex variant of SGD).

So the "Stochastic" part means that it's random.

The stochastic gradient descent works by taking a smaller random part of the training data, called "mini batch", and back propagates (trains) on this. Doing this until the entire dataset is processed once is often called one epoch*.

This is how gradient descent works in a nutshell: Imagine you're going down a U-shaped hill. You're pretty far down in the U-shape, and you want to go further down by jumping. You figure out what direction is "down" for you: and then you jump. But darn it: you jumped too far and you ended up further up on the other side of the U!

That is just a simple example. You are probably working in WAY bigger dimensions, which complicates this analogy a bit.

Anyway this results in the effect that the loss might go up from time to time when you train another epoch. If you are training a lot of epochs and the loss keeps going up, you should check the learning rate (which basically decides how big a "jump" is).

Hope it helps :)

*: There are other ways of defining an epoch, but it all goes in variants of this.


Given unlimited time and a perfect fitness function, could a genetic program solve any problem? Obviously this is hypothetical, but is true? I know "perfect fitness function" is a bit hand-wavy, but I mean it as we have a perfect way to measure the completion of any problem.

Yes.

A totally random algorithm could solve any problem given unlimited time and a perfect fitness function. All you need to do is give the GA some new random population members each generation and you're guaranteed to find the solution eventually. Even if you keep only descendants of the previous generation, setting the mutation rate and number of crossovers high enough could effectively get you random individuals.


Are any organisations using AI to predict weather? This seems like a natural fit, though I've not heard of any, yet.

I would love to know if any MET office, government, military or academic institution has taken all (or sizeable portion of) recorded global weather data for, say, the last 50 years (or since we, as a race, have been using weather satellites) and used it in an AI system to predict future weather.

People have used machine learning models on aspects of weather forecasting, as here: Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting or here Predicting Solar Generation from Weather Forecasts using Machine Learning.  I've been loosely associated with an effort to use ML techniques to predict utility demand from weather data.  But note that these are looking to predict the implications of the weather forecast, not the forecast itself.  Lots of effort goes into refining the physics-based models that underlie our everyday weather forecasts, and AFAIK machine learning hasn't resulted in any "secret sauce" that gives better results than these models provide.


Is it possible to create a named entity recognition system without using POS tagging in the corpus? Is it possible to create a named entity recognition system without using POS tagging in the corpus?

Yes, both of them are different tasks. POS tagging helps NER systems but it is not necessary. You can get features (say BERT/ELMo embedding) for each word in the sentence and train a CRF NER model. This looks like simple example https://www.pragnakalp.com/bert-named-entity-recognition-ner-tutorial-demo/


Will computers be able to understand user emotions? I am researching affective computing. Particularly, I'm studying the part of emotion recognition, i.e. the task of recognising the emotions that are being felt by the user/subject. For example, affectiva can be used to this end. I have concerns not in the validity of these models, but in what we are going to do with them.

What about responding to emotions? Will computers be able to really understand user emotions?

Will computers be able to understand user emotions?

The term Understand is multidimensional, so characterizing the degree of understanding — emotional or otherwise — is a slippery task.  Nonetheless, some forms of AI emotional understanding are possible.

An Interesting Simple Case

Even the embedded programs of velocity sensitive musical keyboards are emotionally aware at a primitive level.  Using the key depression velocity to drive the note parameters of attack, timbre, and volume allows the accomplished user's emotions to pass through the software into the emotional content of the music, which can be experienced by listeners just as with instruments that are not digital in nature.

More Advanced Emotional Capabilities

At the other extreme is a wide array of human emotional capabilities.  One example is the set of complex recognition functions underneath the typical listening skill set of a counselling professional.

The term affective computing was mentioned in the question.  A counselor may note the affect of the counselled, reading tone of voice, facial expression sequences, and body language.  From this, a clearer picture of the internal emotional state of the counselled can be understood.  These recognition and analysis abilities can determine the subject's internal emotional state more accurately and comprehensively than a masterful analysis of a functional MRI or the detection of neuro-chemical metabolites in blood.

There seems to be no theoretical principle that limits computers from mastering the front end of a counselor's skill set, the recognition of emotion to produce a coded sequence of affects.  The degree to which software can interpret the sequence of affects (in combination with concurrent natural language expressions) and determine the emotional condition (and potential market-oriented decision patterns) of the human is yet to be determined.

There are other emotional capabilities beyond the skills of emotional recognition and interpretation.  Compassion is largely the logical integration of ethics in the absence of self-centered motive with the results of emotional recognition and interpretation.  The addition of emotional imitation yields empathy.

The development of such capabilities in automata is more important than the shallow and therefore myopic motives of e-Commerce.  There may be applications for toys, education, entertainment, and remedy for the growing coldness of a technological society.  (Such is discussed further below.)

Functions of recognition, interpretation, the application of rules or meta-rules in the ethical domain to these interpretations, and imitation may well be within the capabilities of digital systems.

Isaac Asimov introduced the possibility of telepathy to the feature set of both the human mind and automata.  It may seem like telepathy is confined to the domain of fiction, however an emoticon can be the tele-form of a component of affect, such as a smile.  In this way, consensual technology assisted telepathy has actually become commonplace.

With the erosion of privacy in a culture with increasing interaction with digital systems by an increasing segment of the public, less consensual telepathic techniques may be developed and mastered.

Deeper Aspects of the Question

The doubt lies in the existence of a soul or some other autonomous and non-deterministic aspect of the human person.  The existence of these things and whether the simulation of them can be accomplished with a Turing machine and practically deployed to the Von Neumann architecture or a collection of them have neither been proven or dis-proven formally.  If autonomous and non-deterministic elements exist in people, then, even if they can be simulated, we cannot infur they can be realized as independently autonomous.

Challenging the implications of the MIT coined term Meat Machine, the notion that humans are capable of intention by fiat is not disproven, yet a pure Turing machine clearly cannot intend in that way.

This is important in the context of the question in that a computer may only be able to simulate frustration when an intention is thwarted by the conditions of life.  Actual frustration may not be possible.

Current Publicly Experienced Progress

One interesting entry point of computer science into the domain of human emotions is the story based memory and reasoning model purported by Roger Schank of Yale U in the 1990s.  Although the machinery used by Amazon underneath the hold of its ever improving ability to recommend movies in a buyer specific way is Company Confidential to Amazon, one must wonder.

Can e-Commerce develop user profiling regarding books and movies to comprehend the story plot or the arc of protagonist's emotional development? Can it do so sufficiently to match user viewing and purchase with other products?  Is that what Amazon is beginning to do?  Short of an unethical Amazon employee, we can only guess.  Certainly, what is transpiring in leading edge e-Commerce has moved beyond word or phrase matching or tracking interest in particular authors, screenwriters, directors, producers, and stars.

If Roger Schank's story based memory and reasoning has penetrated into e-Commerse, emotional analysis of individuals of within public is well underway, since story plot and protagonist developmental arcs are bound in climaxes and setting up climaxes, all related to emotional states.  If not, emotional analysis of groups within the public has been underway since public relations functions entered into the roles of enterprise IT systems.  (I know for a fact that such occurred decades ago.)

Even if the likes of Facebook, Google, and Amazon are attempting to approximate in various ways the emotional states of individuals in real time, the matching of movie plots of protagonist arcs and such pattern matching and naive categorization is far from what a good counselor or a self-actualized friend or family member might do.

Computer systems to which the public is exposed do not yet appear to accurately developing comprehension of the buyer's emotion.  That is one area where even the most progressive interfaces are still shy.  Furthermore, the computer interface itself is still as dry as it was in the late 20th century.

Short of breaches of confidentiality, what capabilities to recognize, compare, analyze, or simulate emotions may exist in proprietary labs is solely a mater of conjecture.

Warm Interfaces

Returning to the question of the general coldness of cybernetic interfaces in a technological society, there is much for the computer science community to learn.  For one thing, the soft skills of actual technical people often leaves much to be desired, so a cultural shift from cold development teams to culturally progressive and warm teams might be necessary.

Furthermore, one who has engaged with various PaaS (personality as a service) devices such as Siri, Google Assistant, Alexa, Cortana, and Microsoft's Office Assistant, Clippy, might find the paperclip personality of Clippy to be the warmer from among the list.  Perhaps the Clippy didn't see a higher level of acceptance because of the disparity between the coldness of a grid of cells in a spreadsheet and the attempted warmth of the animated character was too much, and the little guy tends to get in the way of some portion of the interface, which can be annoying.  (Integration of concepts was poor.)

Nonetheless, the animated style of Clippy is intended to create a mental association with cartoon characters, which is brilliant.  Cartoon character design and storytelling through comic and cartoon media is a creative area of technology that had mastered computer generated warmth long before intelligent assistants began to appear.

One can't help thinking, upon general survey of intelligent assistant history, that the more the system tries to be human, the less human it seems.  Human response to emotional content in a cybernetic interface is like that of color matching or synthesized musical instruments.  If one is attempting to synthesize, the synthesis has to be good.  If it can't be fully convincing, then there is more authenticity in inventing something that is not pretending to be something real.  A caricature of real life objects (like Sponge Bob or Road Runner) avoids exposing an unacceptable margin of error in the imitation of real life. [1]

Our sensitivity in this area is so acute that if a real person, working in a customer service department, acts too mechanically such that we have even the slightest doubt as to whether we are talking to a human being, the impression is negative and we want to hang up and look for the function we wish to initiate on the company's web site.

It is unclear whether the trend for people to become more like cogs in a machine or whether usage patterns will drive companies to design their user interfaces and the mechanisms behind them to exude something like hospitality or congeniality.  Although Jaques Ellul seems to be correct in his proposition that technology has long since been driving humanity rather than the other way around, the DNA nature of the human mind will perhaps force humanity to remain remarkably human.  Thus the evolution of cybernetic interface professionals may continue to seek the meaning of and simulation of warmth.

The problematic, controvertible, and somewhat bizarre pursuit of Turing Testing will probably endure.

Concerns About Emotional Modelling Uses

Whether an emotional recognition, modelling, analysis, and comparison suite, obviously a powerful tool-set, is used for the betterment of humanity or its detriment is a question like that of any other powerful tool-set.  Mass psychology, nuclear science, statistics, and genetics have in common the potential for great good or great evil.

In the context of this question, broadcast or more targeted information dissemination can be characterized as propaganda, which may not necessarily be perilous.  Propaganda was used for women's suffrage and along the path of U.S. independence from Europe.  Elements of broadcast and targeted control have created much geopolitical trouble, a topic outside of the scope of this social network, but pertinent and somewhat obvious, such that further mention is unnecessary anyway.

Individual profiling can be of great good too.  Such can be like a combination of the old library subject indexed Dewey Decimal cards in combination with a knowledgeable librarian when looking for information or items of entertainment or personal expression for borrowing, rental, or purchase.

Of course, profiling in combination with statistics can lead to the ability to obtain covert control over a mass of people, not just for the profit maximization of a company.  An appropriately self-deluded megalomaniac or a collection of them in some kind of secret society could use emotional modelling and comprehensive user profiling for a large sample of national or global population for the injection of emotionally powered systems of belief.  Such a person or group could theoretically succeed in covert and systematic sublimation of liberty to a kind of virtual authoritarian dictatorship.

How technology is used is always a concern.  Transparency and accountability are required features of a society.

Just as in the other areas of science and technology mentioned above, it is up to we who are skilled enough to realize these systems to be intelligent and active elements within our society's accountability mechanisms.  Whether a corporation or governmental department is leading humanity into perdition is an item that can be judged during a job hunt and during the initial work period.

Notes

[1]

This is perhaps the strongest critique of the Turing's Imitation Game philosophy.  Cartoons, comics, and sci-fi creatures demonstrate that people don't seem to care if intelligent beings can be distinguished from humans, and there is no proof that human intelligence is the only or the best example of intelligence.

Along these same lines, heroes are representations of intelligent behavior that epitomize some aspect of humanity that actually doesn't naturally occur in pure form as they do with these protagonists.  Yet actors are branded and marketed as stars because of this idyllic expression of character.  It may be that humans don't really like human intelligence and constantly try to transcend it through fiction, but results in only some public awareness that is never more than an anthropological fantasy.

Even AI is not an imitation game.  It is a transcendence game.  For this reason, the Turing Test is of limited value.


What do the different actions of the OpenAI gym's environment of 'Pong-v0' represent? Printing action_space for Pong-v0 gives Discrete(6) as output, i.e. $0, 1, 2, 3, 4, 5$ are actions defined in the environment as per the documentation. However, the game needs only 2 controls. Why do we have this discrepancy? Further, is that necessary to identify which number from 0 to 5 corresponds to which action in a gym environment?

You can try the actions yourselves, but if you want another reference, check out the documentation for ALE at GitHub.

In particular, 0 means no action, 1 means fire, which is why they don't have an effect on the racket.

Here's a better way:

env.unwrapped.get_action_meanings()


Fitness Function altenatives in Genetic Algorithms for game AI I have created a Gomoku(5 in a row) AI using Alpha-Beta Pruning. It makes moves on a not-so-stupid level. First, let me vaguely describe the grading function of the Alpha-Beta algorithm.

When it receives a board as an input, it first finds all repetitions of stones and gives it a score out of 4 possible values depending on its usefulness as an threat, which is decided by length. And it will return the summation of all the repetition scores.

But, the problem is that I explicitly decided the scores(4 in total), and they don't seem like the best choices. So I've decided to implement a genetic algorithm to generate these scores. Each of the genes will be one of 4 scores. So for example, the chromosome of the hard-coded scores would be: [5, 40000,10000000,50000]

However, because I'm using the genetic algorithm to create the scores of the grading function, I'm not sure how I should implement the genetic fitness function. So instead, I have thought of the following:

Instead of using a fitness function, I'll just merge the selection process together: If I have 2 chromosomes, A and B, and need to select one, I'll simulate a game using both A and B chromosomes in each AI, and select the chromosome which wins.

1.Is this a viable replacement to the Fitness function?

2.Because of the characteristics of the Alpha-Beta algorithm, I need to give the max score to the win condition, which in most cases is set to infinity. However, because I can't use Infinity, I just used an absurdly large number. Do I also need to add this score to the chromosome? Or because it's insignificant and doesn't change the values of the grading function, leave it as a constant?

3.When initially creating chromosomes, random generation, following standard distribution is said to be the most optimal. However, genes in my case have large deviation. Would it still be okay to generate chromosomes randomly?

Is this a viable replacement to the Fitness function?

Sure, the fitness is 1 for the winner and 0 for the loser. You're using some kind of the Tournament selection.

It might be better to use more chromosomes and let A play against B, C, D... and define the fitness as the number of wins. Or not, as such an evaluation is more precise but also more time-consuming.

  ... Infinity ... Do I also need to add this score to the chromosome?

Why should you? The exact value doesn't matter (as it only needs to be big enough), so there's nothing to evolve there. You don't represent the number  of players either, right? Just use common sense.

  ... Would it still be okay to generate chromosomes randomly?

I guess so, but a distribution providing values closer to the expected outcome should be better. This depends on how you mutate them (adding a small random value won't bring you far, multiplying by 1 + small_random_value would).

Alternatively, you can generate values from some fixed interval and scale them up.

Your values are IMHO far too big. Whatever your 5 and 40000 mean, I guess, that 5 and 400 would work the same.


Can the rewards be matrices when using DQN? I have a basic question. I'm working towards developing a reward function for my DQN. I'd like to train an RL agent to edit pixels on an image. I understand that convolutions are ideal for working with images, but I'd like to observe the agent doing it in real-time. Just a fun side project.
Anyway, to encourage an RL agent to craft a specific image I'm crafting a reward function that returns a $N \times N$ dimensional matrix. Which represents the distance between the state of the target image (RGB values for each pixel location) and the image the agent crafted.
Generally speaking, is it better for rewards to be a scalar, or is using matrices okay?

Generally speaking, is it better for rewards to be a scalar, or is using matrices okay?

Rewards need to be scalar, real values to match to standard theory of Markov decision processes (MDPs) and reinforcement learning (RL) methods.
Although it is possible to accumulate matrices in various ways, by e.g. simple matrix addition, and come up with an analog for expected return which would be a weighted sum of matrices, you then get stuck. There is no fixed way to rank matrices and decide whether one is a better result than another. This is a requirement for any learning process that aims to improve at a task - it needs feedback that changes it makes are better or worse related to some reference. As a result, most objective functions and metrics in optimisation use real-valued scalars, which can always be placed into order to decide a highest or lowest value.
This does not prevent you using a matrix representation for your project, if it is a natural fit. To turn it into a usable reward, you will need to convert that matrix into a real-valued metric. Perhaps the L2 norm or other standard measure that summarises the matrix will be good for your task.
It is possible to process multiple scalar rewards at once with single learner, using multi-objective reinforcement learning. Applied to your problem, this would give you access to a matrix of policies, each of which maximised the reward value of one cell within the matrix. It also allows for switching between objectives in a hierarchical manner using a "policy of policies" if some preference for what to achieve changes. It is not 100% clear, but I do not think this is what you want to do.


How to use convolution neural network in Deep-Q? I currently have a grid of pixels 20x20. Each pixel can be red green blue or black. So I have one hot-encoded the pixels giving a 20x20x4 array for each screen.

For my Deep-Q Network, I have attached two successive screenshots of the screen together giving a 20x20x4x2 array.

I am trying to build a Convolutional Neural Network to estimate the Q values but I am not sure if my current architecture is a good idea. It currently is as shown below:

    def create_model(self):
        model = Sequential()
        model.add(Conv3D(256, (4, 4,2), input_shape=(20,20,4,2)))
        model.add(Activation('relu'))
        model.add(Dropout(0.2))

        model.add(Conv3D(256, (2,2,1), input_shape=self.input_shape))
        model.add(Activation('relu'))

        model.add(Flatten())
        model.add(Dense(64))
        model.add(Dense(self.num_actions, activation='linear'))
        model.compile(loss='mse', optimizer=Adam(self.learning_rate), metrics=['accuracy'])
        return model

Is a 3d convolution a good idea?
Is 256 filters a good idea?
Are the filters (4,4,2) and (2,2,1) suitable?
I realise answers may be highly subjective but I'm just looking for someone to point out any immediate flaws in the architecture.

Is a 3d convolution a good idea? Is 256 filters a good idea? Are the filters (4,4,2) and (2,2,1) suitable?

It's not so much that answers are subjective, but you are performing an experiment, and this should be driven by results. If you can find something published about a similar environment that might help you narrow down your choices.

That said, intuitively you don't gain much going from 2d to 3d convolutions when one of your dimensions is only sized 2. Purely from gut feeling I would suggest simply concatenating your two frames by having 8 channels instead of 4, and use 2d filters. This is simple to enough try and compare though, so perhaps you can do both.

You will likely want to explore deeper networks that have fewer initial filters and build up over a few more than two convolutional layers.

Definitely try your network without the dropout layer. I have not had much luck with using dropout in DQN, and I have seen others having similar problems. I am not sure what the exact issue is.

The accuracy metric won't do you much good on a regression task, so you can drop that and just use MSE loss. Bear in mind that training loss is a less useful metric overall in RL, since the prediction target (action value) is continuously changing as the policy changes. Low loss values mean that the value predictions are self-consistent, they don't necessarily mean that the learning has converged to an optimal policy.


How can I minimise the false positives? I have 50,000 samples. Of these 23,000 belong to the desired class $A$. I can sacrifice the number of instances that are classified as belonging to the desired class $A$. It will be enough for me to get 7000 instances in the desired class $A$, provided that most of these instances classified as belonging to $A$ really belong to the desired class $A$. How can I do this?

The following is the confusion matrix in the case the instances are perfectly classified.

[[23000   0]
 [  0 27000]]

But it is unlikely to obtain this confusion matrix, so I'm quite satisfied with the following confusion matrix.

[[7000   16000]
 [  500 26500]]

I am currently using the sklearn library. I mainly use algorithms based on decision trees, as they are quite fast in the calculation.

I think you're looking for the minimization of false positives, that is, the instances that are classified as belonging to the desired class (the positive part of false positives) but that do not actually belong to that class (the false part of false positives). In practice, given your constraints, you may want to maximize the precision, while maintaining a good recall.

In this answer to the question How can the model be tuned to improve precision, when precision is much more important than recall?, the user suggests performing a grid search (using sklearn.grid_search.GridSearchCV(clf, param_grid, scoring="precision")) to find the parameters of the model that maximize the precision. See also the question Classifier with adjustable precision vs recall.


Neural network for detecting individual human face In past few weeks, I have learned a lot about Neural Networks. Now, I am looking forward to create a Neural Network program that can recognize individual human faces. I tried searching it online but was able to find only small pieces of information.
What are the steps for implementing such a program from scratch?

I am assuming that you are new to all this. You can start with making a basic human face detection. Train the program to detect a human face with very good accuracy. This will help you to get familiar with the coding ground related to image processing and basic machine learning.
After that train your program to identify faces of only 2-3 people. Trying for too many in the beginning won't be a good idea. Test your program's accuracy in different situation, with a different number of crowd, etc.
If it's working fine then you can train your program for more individuals. In addition to this leave room for learning from experience in your code. Some codes only learn once and they implement the same thing for their whole life. A nice example of this kind is OCR. If a face is detected wrongly OR your program detects a new face about which it doesn't know anything. Then you should be able to tell the program and it should include that in its database. I think some form of reinforcement learning will help. Not much sure about it though.
Now the Implementation
I will highly recommend you to learn python and get familiar with OpenCV. You can think of OpenCV as a collection of libraries. I find them very helpful for image processing and machine learning. Another good thing about it is that you can import OpenCV in python, Java or C++ according to your need.
OpenCV has an inbuilt function that allows it train a neural network for positive and negative images. The success of your program depend highly on your choice of positive and negative images, so choose them wisely. The result of the training is stored as a haar cascade file. This cascade file can be used in your program to use the trained data and function accordingly.
For basic human face detection, you can find the cascade file online and implement a code like this

  face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml');

For detecting individual faces you will need to make different classes. The number of classes will represent the number of individual faces you want to detect in the beginning.
You can find OpenCV tutorials here.


Do we train a logistic regression model using a dataset that is 3 times bigger than the validation dataset? Suppose we have a data set $X$ that is split as $X_{\text{train}}$, $X_{\text{val}}$ and $X_{\text{test}}$ and the outcome variable is binary. Let's say we train three different models (logistic regression, random forest, and a support vector machine) using $X_{\text{train}}$. We then get predictions for $X_{\text{val}}$ using each of the three models.

In stacking, is it correct to say that we train a logistic regression model on a data set of dimension $|X_{\text{val}}| \times 3$ with the predicted values and actual values of the validation set? This logistic regression model is then used to predict outcomes for data in $X_{\text{test}}$?

The predictions of each of your initial models will become a feature to feed the meta learner. If you use $n$ initial models, then for each example you will feed the meta learner $n$ features, each feature being a prediction from one of the initial models.

Note that this doesn’t mean the size of your dataset increases. Instead, each member of $X_{val}$ will be represented by $n$ features, with the $k$th feature being equal to the initial prediction of the $k$th initial predictor.

This medium link recommends a package that can help you build a pipeline.

Note: in a sense if you are caching the intermediate predictions of each initial predictor, you’ll end up with $n$ times as many data points as there are entries in $X_{val}$. But these are simply each going to be a feature of another data point, so the dataset hasn’t really increased in size.


How to make DNN learn multiplication/division? A single neuron with 2 weights and identity activation can learn addition/subtraction as the 2 weights will converge to 1 and 1 (addition), or 1 and -1 (subtraction).

However, for multiplication and division, it's not that easy. Can a single neuron learn multiplication or division? If not, how many layers of DNN can learn these?

In reallity any continous function on a compact can be approximated by a neural network having one hidden layer with a finite number of neurones (This is the Universal Approximation Theorem). Thus you only need one hidden layer to approximate the multiplication on a compact, note that you need to apply a non linear activation on the hidden layer to do this.


What do the numbers in this CNN architecture stand for? So I've got a neural net model (ResNet-18) and made a diagram according to the literature (https://arxiv.org/abs/1512.03385).

I think I understand most of the format of the convolutional layers:
filter dims ,conv, unknown number ,stride(if applicable)

What does the number after 'conv' in the convolutional layers indicate? is it the number of neurons in the layer?

bonus q: this is being used for unsupervised learning of images, i.e the embedding output a network produces for an image is used for clustering. Would this make it incorrect for my architecture to have an FC layer at the end (which would be used for classifcation)?

This number refers to the number of kernels (or feature maps) that are convolved with the input. So, for example, in the first convolutional layer, $64$ $3 \times 3$ kernels are convolved with the image.

The ResNet presented in Deep Residual Learning for Image Recognition is used for image classification. Furthermore, note that your diagram already contains a fully connected layer at the end.


Will artificial super-intelligence evolve to have selfishness inherent in biological systems? A lot of experts have expressed concerns about evil super intelligence. While their concerns are valid, is it necessary, what are the chances or how the artificial super-intelligence will evolve to have selfishness and self protecting desires inherent in biological systems? Is there any work which comments on this line of inquiry?

AI will only "evolve" selfishness if it "evolves" in a competitive environment and has certain human-like faculties.

Self-protecting desires on the other hand are logical consequences of having any goal at all. After all, you can't reach your goal if you are destroyed.

The concern of "evil" super intelligences isn't that they literally turn evil and selfish and cruel. Those are human qualities.

Instead a superintelligence that has a certain goal, will logically pursue subgoals that help it to reach the ultimate goal. Such subgoals will be resources, power, safety.

So it will amass power and resources to reach its goal and exterminate any threat to its existence as long as its goal hasn't been reached, without being selfish at all.


How can alpha zero learn if the tree search stops and restarts before finishing a game? I am trying to understand how alpha zero works, but there is one point that I have problems understanding, even after reading several different explanations. As I understand it (see for example https://applied-data.science/static/main/res/alpha_go_zero_cheat_sheet.png), alpha zero does not perform rollouts. So instead of finishing a game, it stops when it hits an unknown state, uses the neural network to compute probabilities for different actions as well as the value of this state ("probability of winning"), and then propagates the new value up the tree.

The reasoning is that this is much cheaper, since actually completing the game would take more time then just letting the neural network guess the value of a state.

However, this requires that the neural network is decent at predicting the value of a state. But in the beginning of training, obviously it will be bad at this. Moreover, since the monte carlo tree search stops as soon as it hits a new state, and the number of different game states is very large, it seems to me that the simulation will rarely manage to complete a game. And for sure, the neural network can not improve unless it actually completes a significant number of games, because that is only real feedback that tells the agent if it is doing good or bad moves.

What am I missing here?

The only plausible explanation I can come up with is:  If the neural network would be essentially random in the beginning, well then for sure the large number of game states would prevent the tree search from ever finishing if it restarts as soon as it hits a previously unknown game state, so this can not be the case. So perhaps, maybe even if the neural network is bad in the beginning, it will not be very "random", but still be quite biased towards some paths. This would mean that the search would be biased to some smaller set of states among the vast number of different game states, and thus it would tend to take the same path more than once and be able to complete some games and get feedback. Is this "resolution" correct?

One problem I have though with the above "resolution", is that according to the algorithm, it should favor exploration in the beginning, so it seems that in the beginning it will be biased towards choosing previously not taken actions. This makes it even more seem like the tree search will never be able to complete a game and thus the neural net would not learn.

You're right that AlphaGo Zero doesn't perform rollouts in its MCTS. It does complete many, many games, though.

Realize that AlphaGo Zero only iterates MCTS 1,600 times before taking an action. The next state resulting from that action becomes the root of future search trees. Since a typical game of Go only lasts a few hundred moves, the board will very quickly reach a terminal state.

None of this is dependent on the initial performance of the neural network. The neural net can be incredibly bad; actions/moves will still be taken at the same frequency. Of course, since AlphaGo Zero trains with self-play, one of its two selves in a game will usually be the winner (ties are possible). So the neural net will improve over time.

I recommend going over the "Self-Play Training Pipeline" section of the paper.


How is few-shot learning different from transfer learning? To my understanding, transfer learning helps to incorporate data from other related datasets and achieve the task with less labelled data (maybe in 100s of images per category).
Few-shot learning seems to do the same, with maybe 5-20 images per category. Is that the only difference?
In both cases, we initially train the neural network with a large dataset, then fine-tune it with our custom datasets.
So, how is few-shot learning different from transfer learning?

They use the same techniques, but study different problems.
Transfer learning always does not imply that the novel classes have very-few samples (as few as 1 per class). Few-shot learning does.
The goal of transfer learning is to obtain transferrable features that can be used for a wide variety of downstream discriminative tasks. One example is using an ImageNet pretrained model as an initialization for any downstream task, but note that we need to train on large amounts of data on those novel classes for the model to be suitable to that task.
Note that you can't finetune an ImageNet classifier on few examples of COCO and expect it to generalize well, because it won't. It wasn't explicitly optimized for few-sample learning.
In few-shot learning, our aim is to obtain models that can generalize from few-samples. This could be transfer learned (with certain changes to the usual transfer learning scenario), or it could be meta-learned. It might not need both, it could just be augmented with data from the novel classes during the test time, and a classifier could be trained from scratch.


Is there a way to ensure that my model is able to recognize an unseen example? My question is more theoretical than practical. Let's say that I am training my cat classifier with a dataset that I feel is pretty representative of cat images in general. But then a new breed of cat is created that is distinct from other cats and it does not exist in my dataset. My question is, is there a way to ensure that my model is still able to recognize this unseen breed, even though I didn't know it would come into existence when I originally trained my model?

I have been trying to answer this question by intentionally designing my dev and test sets such that they contain examples that are quite distantly related to those that exist in the training set (think of it like intentionally leaving out specific breeds of cats from the training set). The results are interesting. For example, slight changes to parameters can dramatically change performance on the distantly related test examples, while not changing performance very much for the more closely related examples. I was wondering if anyone has done a deeper analysis of this phenomenon.

The comments already are giving you some good tips about how to improve what your model recognizes, but I think your question goes above that asking if there's a way to ensure that it will always recognize the cats.

The short answer is "no".

The slightly longer answer is "yes, but cheating".

Regardless, there are a lot of steps you might take to improve the generalization aspect of your model.

Long answer:

A drama: cat classification in three acts

Act I: Cat as texts

Let's start with an example. Say that your model is trained with these inputs, and learns to correctly recognize them as a cat or not a cat:

cat → yes!
Cat → yes!
ferret → no
cat. → yes!
Cat! → yes!
Three MC's and one DJ → no

Your goal is to train your model so that every new variation, even unseen ones, will correctly be identified.

With a good level of generalization, your model will correctly classify new inputs that it has never seen before:

skunk → no
cat? → yes!
dog → no
CAT → yes

With this scenario, let's say the model now finds this:

kat → ?

Is that a "cat" misspelled? Is that short for Katherine? What should the model do?

Act II: But surely this doesn't happen in real life

Leaving the analogy for a bit, will your model that's looking at domestic cats properly accommodate for Savannah Cats, or will it consider them out? (They kind of look like cheetahs.) What about Sphinx cats? (They look like raw chicken to me.) Elf cats? (They look like bats.) This is just an example, but you can probably figure out more.

And the reason behind this problem is that the distinction itself between different classifications (in real life) is not binary, but rather a transition between "yes, that's a textbook cat" and "that's a chair". Your model will output binary decisions (maybe accompanied by a confidence interval, but even with it, you'll make the call into deciding if it's a cat or not).

Setting specific boundaries will help. You can define that your model will only detect domestic cats, maybe no bigger than a certain size, only of certain colors, etc... This is limiting what the model will correctly recognize as a cat when we (humans) might disagree. For instance, I would still argue that flourescent cats are still cats.

Going back to the simple text analogy, this is similar to deciding that to be detected as a cat, it has to start with a "c". So now you've discarded ¡Cat!.

In this way, it's not possible to ensure (notice the word) that your model will detect all of these unknown variations. There will be always some room for error that needs to be accepted, as long as the errors are infrequent or rare enough that they can be accepted as a regular part of the model.

Act III: Concept drift, a cautionary tale

Finally, the problem becomes even harder as we might be dealing with concepts that change over time, outside of the knowledge of the model, and outside of the knowledge of the person that supervised the model learning.

As and the breeds of cat changes, your model will have to accommodate by what we (users of the model) consider a valid definition of cat. Which might change in really unexpected ways and not really "look" like a cat. And since your model can only learn from what "looks" like a cat, it's always held in a disadvantaged position.

This will happen with almost any machine learning model that is approximating a result, regardless of the technique/algorithm. Approximations include a level of error because reality is usually complex in ways that we either don't know about or that are too computationally expensive.


Alternative to sliding window neural network (was: Object detect (or) image classification at specific locations in the frame) Recent advances in Deeplearning and dedicated hardware has made it possible to detect images with a much better accuracy than ever. Neural networks are the gold standard for computer vision application and are used widely in the industry, for example for internet search engines and autonomous cars. In real life problems, the image contains of regions with different objects. It is not enough to only identify the picture but elements of the picture.

A while ago an alternative to the well known sliding window algorithm was described in the literature, called Region Proposal Networks. It is basically a convolution neural network which was extended by a region vector.

Problem that I am trying to solve:

In a given video frame, I want to pick some region of interests (literally), and perform classification on those regions.

How is it currently implemented

Capture the video frame
Split the video frame into multiple images each representing a region of interest
Perform image classification(inference) on each of the image (corresponding to a part of the frame)
Aggregate the results of #3

Problem with the current approach

Multiple inferences per frame.

Question

I am looking for a solution where I specify the locations of interest in a frame, and inference task, be it object detection (or) image classification, is performed only on those regions.Can you please point to me the references which I need to study (or) use to do this.

There are many different problems in computer vision. Four of them are well described by the top image here:

Classification: Given an image, say what is on it (a single thing)
Classification+Localization: Given an image, say what is on it and draw an axis-aligned bounding box (AABB) around it
Object detection: Given an image, draw AABBs around every object and classify those objects
Semantic segmentation: See Survey of semantic segmentation
Instance segmentatation: Like semantic segmentation, but if there are multiple cats then they should be recognized as different objects.

Your question seems to be about object detection. The relevant papers here are:

SSD: Single Shot MultiBox Detector: Code
You Only Look Once: Unified, Real-Time Object Detection (YOLO): Website with video
YOLOv3

If you actually already have the regions, then you can simply perform classification on them. When you pad / scale / crop them, you can batch-predict them.


What is the impact of scaling the features on the performance of the model? I am trying to generate a model that uses several physicochemical properties of a molecule (including number of atoms, number of rings, volume, etc.) to predict a numeric value $Y$. I would like to use PLS Regression, and I understand that standardization is very important here. I am programming in Python, using scikit-learn.

The type and range for the features varies. Some are int64 while others are floating point numbers. Some features generally have small (positive or negative) values, while others have a very large value. I have tried using various scalers (e.g. standard scaler, normalize, min-max scaler, etc.). Yet, the R2/Q2 are still low.

I have a few questions:

Is it possible that by scaling, some of the very important features lose their significance, and thus contribute less to explaining the variance of the response variable?
If yes, if I identify some important features (by expert knowledge), is it OK to scale other features but those? Or scale the important features only?
Some of the features, although not always correlated, have values that are in a similar range (e.g. 100-400), compared to others (e.g. -1 to 10). Is it possible to scale only a specific group of features that are within the same range?

In general, algorithms that exploit distances or similarities (e.g. in the form of scalar product) between data samples, such as k-NN and SVM, are sensitive to feature transformations. We do feature scaling to make our model robust to outliers and make an initial impact of every feature on the model will be roughly similar

Graphical-model based classifiers, such as Fisher LDA or Naive Bayes, as well as Decision trees and Tree-based ensemble methods (RF, XGB) are invariant to feature scaling, but, still, it might be a good idea to rescale/standardize your data.

You should explore your data more carefully, find the outliers, apply transformation if needed.
Not sure if it is a good idea
You can apply different preprocessing techniques like MinMaxScaller, Rank,  Log transform, Extracting square root, StandartScaller and etc.


What approach should I use to detect faces in video game footage? I have set myself the challenge of detecting the locations of players/bots in videos of a well known first person shooter game (this is for a youtube series I'm planning on doing). I'm not sure which AI approach I should apply to this problem - I'm a complete novice at this!

My first thought was that the face/head seems to have the most detail so I could train a convolution neural network on images of sprite heads and general background - however this seem not to work too well, I've certainly not exhausted different network architectures/typologies but it wasn't learning all that well.

My second approach was to use a HAAR cascade. This seemed to be an obvious choice since it's fast and good at detecting objects (rather than multi-classifying). However my cascade stops after 5 or 6 stages (using OpenCV) as it seems to have reach a great accuracy, but it doesn't detect when I feed it the training images, let alone other images.

I also looked into pedestrian detection and got a stock version of that working. However this seemed to struggle when/if the sprites are crouching or in unusual positions (and it isn't great on standing sprites tbh).

So, is there a branch of machine learning/AI that is more applicable to this problem? If not which should I continue to work on?

At first, you can find lots of information as pedestrian detection. As you are trying to localize game characters, the face is not the best option. You need to look for the character in general.

About HAAR Cascades, the algorithm is one of the fastest face localization solutions in the market. The reason is, it applies all the feature classifications layer by layer, by starting from the wider feature. So, if it fails, it does not spend the time to check computationally intensive features. It was good until DNN overcome its success rate. However, it is not the best approach for recognizing game characters/pedestrians.

Also, it seems like you are overfitting the cascade so it stops and does not learn anything valuable. You can search methods about how to reduce overfit problem.

In 2005, a new method has been proposed, HOG (Histogram Of Gradients). You can use this one and classify output features to get what you desire. If you would like to go for deep learning version, I would suggest you to investigate how DNN is working, which kind of input images you need, what are the localization networks (i.e. YOLO, Faster R-CNN), how they are working.


What are the minimum requirements to call something artificial neural network? I've found this short Python code which implements neural network in 11 lines of code:

X = np.array([ [0,0,1],[0,1,1],[1,0,1],[1,1,1] ])
y = np.array([[0,1,1,0]]).T
syn0 = 2*np.random.random((3,4)) - 1
syn1 = 2*np.random.random((4,1)) - 1
for j in xrange(60000):
    l1 = 1/(1+np.exp(-(np.dot(X,syn0))))
    l2 = 1/(1+np.exp(-(np.dot(l1,syn1))))
    l2_delta = (y - l2)*(l2*(1-l2))
    l1_delta = l2_delta.dot(syn1.T) * (l1 * (1-l1))
    syn1 += l1.T.dot(l2_delta)
    syn0 += X.T.dot(l1_delta)

I believe it may be a valid implementation of neural network, but how do I know?

In other words, is just creating bunch of arrays which compute the output on certain criteria and call them layers with synapses does it make proper neural network?

In other words, I'd like to ask, what features/properties makes a valid artificial neural network?

If you pick up a textbook on Neural Networks, you'll find that the simplest examples shown are ones that just implement an AND gate or something.  They're trivial, probably fewer lines of code than what you have there.  The bar to be an "artificial neural network" is pretty low... it certainly isn't the case that ANN's must be incredibly complicated with thousands of lines of code, and many layers, or even many "neurons" total.

Basically, if something is setting up at least one "neuron" with multiple inputs, and using some kind of weighting function to generate an output from those inputs, it's a valid ANN. It might be a really simple example of an ANN, but it's still an ANN.

Remember what Geoffrey Hinton says in his Coursera class - (paraphrased) "We don't pretend that the things we're building really work the way the brain does, we're just taking the brain as loose inspiration for an approach that we've found works".


Why do we need learning in unsupervised learning? I am not clear with the concept that an unsupervised model learns. We are giving an input and output to the supervised model, so that it can generate a particular value, pattern or something out of it which can be used to categorize something in the future. By contrast, in unsupervised learning, we are clustering, so why do we need learning?

Can anyone detail me with some real-world examples?

Imagine you have a dataset of people who have cancer.
You have information about their age, physique, diagnosis, treatments, and results.

Using this data, you want to prescribe a set of treatments for a new patient, P.

Obviously, if there is someone in the dataset that has very similar traits as P and had a positive result with their treatments, you could prescribe the same set of treatments. However, this is incredibly unlikely and becomes more infeasible as more information about P is observed (e.g. Has brown hair and hates pasta).

A better option is to cluster the dataset into groups that have positive outcomes for treatment results. For example, perhaps patients with lung cancer who smoke and are given treatment A do better than patients with lung cancer who didn't smoke and are given the same treatment A. These patients should then be divided based on this outcome.

Once these different clusters are found, patient P can be evaluated against each of the clusters and a set of treatments can be prescribed (e.g. Most of the treatments from cluster A, but 1 treatment from cluster B).

Unsupervised learning is the method of finding these clusters, which helps find structure to the data to better answer questions.


Does humanism grant moral consideration to sentient artificial general intelligences? Some formulations of humanism already grant moral consideration to sentient non-human animals (e.g. https://humanists.international/what-is-humanism/). Does humanism also extend to granting rights to AGIs, should they become sentient?

Sentientism is a closely related philosophy that makes this explicit https://en.wikipedia.org/wiki/Sentientism.

Does humanism also extend to granting rights to AGIs, should they become sentient?

I think that is going to be something that will divide self-identifying humanists into a few different groups, based on understanding about what AGI is.

The front page you link includes "aiming toward the welfare and fulfillment of living things", which does not explicitly include non-living sentient things. However, the humanist philosophy naturally extends to granting rights and protections based on reasoning and scientific evidence to anything that can be argued as needing them.

The big problems are:

Recognising and measuring sentience.
Attaching moral value to events that happen to artificial entities, where subjective measures such as pain, stress, happiness, fulfilment might not be relevant.

These measurement problems already cause a lot of variability in moral stances towards farming animals for instance. These biological creatures share enough with humans that even though we don't fully understand the nature of their subjective experience, it is possible to extrapolate with some backing from scientific studies. For example, we can measure whether a creature suffers stress or pain and reacts similarly to a human, and even if we don't understand what that means subjectively (what it feels like to be such an animal), a combination of Occam's Razor and Precautionary Principle can be argued:

Occam's Razor the simplest interpretation is that if an animal is based on same cellular structure, has same brain regions for experiencing pain and reward, uses same stress hormones etc, that its subjective experience is likely somehow comparable to a humans. This allows us to apply empathy to an animal's situation with a guess that doing so is not 100% inaccurate.
Precautionary Principle the consequences for inhumanely treating a sentient creature are morally worse than the waste of humanely treating a non-sentient creature (provided doing the latter is not causing undue suffering to some other thing). So faced with a lack of precise knowledge, there is still a strong rational ethics argument for extending rights and humane treatment if there is any uncertainty.

When it comes to artificially created entities, we lose the first argument of Occam's Razor, and can only get it back with a better understanding of what sentience actually is, and which parts of it we should value. We need a much firmer theory here than with animals, because there is so little shared between our construction and that of a human-built machine. In addition, just as there are many different kinds of organism which exhibit different amounts of apparent intelligence and possible sentience, there will be different versions of AGI, probably starting with purpose-built non-narrow AIs that combine features that are practically useful and make them seem intelligent (parsing human speech and responding appropriately, which Google Home, Alexa, Siri already do to some depth) with some common-sense understanding of broad task portfolios. These things are near future, and most developers who work on them would not consider them sentient - however, they will be designed to act as if they were, because that makes them practical and usable, which in turn will make it much harder to argue Occam's Razor due to similarity in behaviour.

We still keep the second argument, but it is confounded by not understanding what the analogs are for humane and inhumane when dealing with artificial entities. It is unlikely that anyone would have a problem with switching off a running instance of the next 2 or 3 generations of Siri assistants - but at what point does that become deliberate incapacitation of a sentient being, and does it even matter if the system is designed to be switched on and off, and expresses no particular desire to be in either state?

In summary, the humanist philosophy could extend to cover the welfare of artificial general intelligences, but it is far from clear whether it can go there in practice. Until we have more complete theoretical models of AGIs, or practical working devices, then it is premature to consider it. Most debate is going to be based on imagined qualities of AGI, and any two people discussing this are likely to be starting from different assumptions about what those are.


Are there any computational models of mirror neurons? From Wikipedia:

  A mirror neuron is a neuron that fires both when an animal acts and when the animal observes the same action performed by another.

Mirror neurons are related to imitation learning, a very useful feature that is missing in current real-world A.I. implementations. Instead of learning from input-output examples (supervised learning) or from rewards (reinforcement learning), an agent with mirror neurons would be able to learn by simply observing other agents, translating their movements to its own coordinate system. What do we have on this subject regarding computational models?

This article gives a description of mirror neurons in terms of Hebbian learning, a mechanism that has been widely used in AI. I don't know whether the formulation given in the article has ever actually been implemented computationally.