Identifying if an image contains an object with very small (five image) training data set Let's suppose I have 5 images, all of which I assure you are of the same item, but from various angles and perhaps different lighting conditions. I now supply you with an additional image, and I want a score of how likely this image is to contain the item depicted in the first five pictures.

Let us suppose that the item isn't too complex. It won't be a pile of fabric with a pattern on it, dropped several different ways, and won't be a keychain with keys in different conformations. It will also be more complex than just a blue ball shot from different angles. How might you approach the problem of scoring this image?

I would do the following:

Add some images without the item present to your training data. Ideally some other item taken against similar backgrounds, lighting etc. And maybe some of just the background.
Obtain a pre-trained neural network CNN model, such as AlexNet, VGG-19 or Inception v5.
Load the neural network model, remove the very last layer (typically a 1000-way softmax), and replace with a brand new layer for classifying whether your object is present or not. Maybe use a 2-way softmax for "my object" vs "unknown object".
Freeze all the layers other than your own (most NN libraries will have a way to do this, or you could just run the old network truncated to extract a feature vector from penultimate layer.
Train using your 5 images. This kind of limited training is often called "fine tuning".
Find some way to test your classifier before relying on it. You may have to do a leave-one-out test where you train your network 5 times, each with 4 of the images, and see whether it correctly identifies the 5th one vs a few images without the item in it. Even with that approach, with such a small sample, you will have only the vaguest idea of how reliable your new classifier is.

You'll do a lot better if you can obtain more images to train and test with. If you get enough images (both positive and negative), you can consider replacing more layers before fine tuning. Otherwise you are heavily relying on your object being similar enough to something from the ImageNet collection that a classifier trained on that will extract useful high-level features.


Binary vector expected value Raul Rojas' Neural Networks A Systematic Introduction, section 8.2.1 calculates the variance of the output of a hidden neuron.

Raul Rojas says that "for binary vectors we have $E[x_i^2] = \frac{1}{3}$" where $x_i$ is the input value transported through each edge to a node.

I don't quite get how he reaches this result.

Thank you for your time :)

Some lines above the author says

  By the law of large numbers we can also assume that the total input to the node has a Gaussian distribution

hence we can assume $X \sim \mathcal{N}(0,1)$ with the $X$ domain being continuous

Then he says the input vector is assumed to be binary which changes the domain from continuous to discrete so we can discretize it assuming $-1 \le X \le 1$ is mapped into zero and $X< -1 $ and $X > 1$ are mapped to 1

Finally according to the 68-95-99.7 Rule we can compute

$$ E(X^2) = P(X=1) \cdot 1^2 + P(X=0) \cdot 0^2 = 0.32 $$

Finally probably the author rounds this up to $\frac{1}{3} \simeq 0.33$


Am I overfitting my GAN model? I'm training a DCGAN model on a 320x320 dataset of images and after an hour of training the generator started to generate (on the same latent space noise as during training) images that are identical to the dataset. For example, if my dataset is images of cars, I should expect to see unexisting designs of cars, right? Am I understanding this wrong? I know this is a very general question but I was wondering if this is what should happen and if I should try on different latent space values and then see proper results and not just copies of my dataset?

It might be that your dataset of images is to small. Your discriminative network might hardlearn these images at which point your generative network can only produce good images if it copies the same images of your dataset.


Can someone help me to understand the alpha-beta pruning algorithm? I understand the minimax algorithm, but I am unable to understand deeply the minimax algorithm with alpha-beta pruning, even after having looked up several sources (on the web) and having tried to read the algorithm and understand how it works.

Do you have a good source that explains alpha-beta pruning clearly, or can you help me to understand the alpha-beta pruning (with a simple explanation)?

Suppose that you have already search a part of the complete search tree, for example the complete left half. This may not yet give you the true game-theoretic value for the root node, but it can already give you some bounds on the game-theoretic value that the player to play in the root node (let's say, the max player) can guarantee by moving into that part of the search tree. Those bounds / guarantees are:

$\alpha$: the minimum score that the maximizing player already knows it can guarantee if we move into the part of the search tree searched so far. Maybe it can still do better (get higher values) by moving into the unsearched part, but it can already definitely get this value.
$\beta$: the maximum score that the minimizing player already knows it can guarantee if we moves into the part of the search tree searched so far. Maybe it can still do better (get lower values) by moving into the unsearched part, but it can already definitely get this value.

The intuitive idea behind alpha-beta pruning is to prune chunks of the search tree that become uninteresting for either player because they already know they can guarantee better based on the $\alpha$ or $\beta$ bounds.

For a simple example, suppose $\alpha = 1$, which means that the maximizing player already has explored a part of the search tree such that it can guarantee at least a value of $1$ by playing inside that part (the minimizing player has no options inside that entire tree to reduce the value below $1$, if the maximizing player plays optimally in that part).

Suppose that, in the current search process, we have arrived at a node where the minimizing player is to play, and it has a long list of child nodes. We evaluate the first of those children, and find a value of $0$. This means that, under the assumption that we reach this node, the minimizing player can already guarantee a value of $0$ (and possibly get even lower, we didn't evaluate the other children yet). But this is worse (for the maximizing player) than the $\alpha = 1$ bound we already had. Without evaluating any of the other children, we can already tell that this part of the search tree is uninteresting, that the maximizing player would make sure that we never end up here, so we can prune the remaining children (which could each have large subtrees below them).


RL: Encoding action conditioned on previous action I have a card game where on a player's turn, the player sequentially draws two cards. Each card may be drawn from another player's discard stack (face up), or from the deck (face down).
Thinking how to encode this into an action space, I could naively assume the two draws are independent. The action space would simply be a binary vector of 2 * (1 + (number_of_players - 1)), which I could post-filter to limit for empty draw piles (and can't draw from own pile).
However, when playing the game myself, I noticed that it's sometimes advantageous to draw the initial card from the deck, then select the draw pile for the second card based on the value of the first one drawn. But how would this be encoded into an action space? Would it be better to think of these are two separate actions, even thought they are part of the same "turn"?

It is hard to say for certain without knowing full details and results of experiments.
However, if the game allows for splitting decisions up, it will likely be better for the agent to take advantage of extra knowledge of the value of any previously hidden card just taken from the draw pile.
In general, if each player decision is taken sequentially, resulting in changes to state, then it is a separate action on a separate time step according to the MDP theoretical model used in reinforcement learning (RL). You might want to describe/notate the time steps differently so that they match how the game play proceeds. However, for the purposes of RL, each decision point should be on a new time step, and should result in a new state, new value estimates etc.
Similarly, whether or not the current choice is the player's first card or second card to be drawn needs to be part of the state. This detail of the state might already be covered by the number of cards in the player's hand, if logically the number of cards is always the same at each stage. However, if hand size can vary for other reasons, it is worth adding an explicit flag for "first draw choice" or similar so that the agent can use the information.
You have some freedom for encoding the action space. If drawing cards is the only possible action in this game at all stages, then a binary output vector of 1 + (number_of_players - 1) dimensions would be suitable. Other encodings may work well too, it depends if there is any logical structure to the choices or some derived data that encodes useful game information.
It may be useful to arrange the action choices so that the index for drawing from each player's discard pile is considered relatively to the current player's turn. That is, instead of actions being arranged $[draw, discard P1, discard P3, discard P4, discard P5]$ for P2, they would be arranged $[draw, discard P3, discard P4, discard P5, discard P1]$ and for P3 would be different: $[draw, discard P4, discard P5, discard P1, discard P2]$ . . . that would inherently allow for the cyclical nature of turns. State representation would need to similarly rotate knowledge about each player to match this. You might not need to do this, but I would recommend it for games where there is a lot of common logic regarding action choices relative to turn position that you could take advantage of. The opposite would apply (and you would use absolute player positions) if there were important differences throughout the game between being P1, P2, P3 etc.


How could human behavior and relationships be implemented? One of the most compelling issues regarding AI would be in behavior and relationships.

What are some of the methods to address this?  For example, friendship, or laughing at joke?  The concept of humor?

In general, what you are describing implies a hierarchical sequence model, in which mannerisms adapt to the regime or paradigm in effect.  Expressive modalities are how we recognize the operative context from the behaviors of other agents.  For an artificial agent, avoiding un-canniness would involve clustering the factors underlying the classification of discourse contexts, tagging them with corresponding factor models for manner in a way which closely aligned to the classifications which human agents make.  This involves developing an adequate latent representation for both spaces, which is likely to involve quite a lot of training data, or some clever transfer learning in conjunction with one-shot techniques (generally, taking samples as modes).

When the context becomes one of friendship, for example, the style factors of expression should extend to factors of trust, collaboration, disclosure, sympathy.  In order to implement these, layered abstractions in the representation of behaviors will need to be crafted, presumably by training.  In order to align these learned categories to human categories, which I suppose to be essential to emotional fluency, exploiting the structure of corpus semantics - including distributional characteristics of the linguistic labels and inferences - when estimating loss gradients seems a natural strategy, which exploits cultural learning.

These comments are necessarily speculative, as no such systems are current, to my knowledge.  At least, not in any significant degree of maturity or application.  Certainly other approaches are conceivable. I am just extrapolating plausible strategies, in the context of current technologies, strategies for implementing the kinds of behaviors you describe in a useful way.

More, specifically, I am considering the question as a manifold-learning problem, and hence requiring a representation model in which the inputs and outputs vary over state-spaces, with the behavior being learned as a mapping between those spaces.  Each of those components (the input space representation and its natural topology, the output space representation likewise, the mapping between them, and the sequence model hierarchy which extrapolates from the manifold to a control process emsemble) has its own peculiar challenges, and the challenges to creating an adequate implementation are a confound of those.  Still, it seems more a question of time and money than of notional feasibility.


If $\alpha$ decreases over time, why is Q-learning guaranteed to converge? Q-Learning is guaranteed to converge if $\alpha$ decreases over time.
On page 161 of the RL book by Sutton and Barto, 2nd edition, section 8.1, they write that Dyna-Q is guaranteed to converge if each action-state pair is selected an infinite number of times and if $\alpha$ decreases appropriately over time.
It seems that it would be better if $\alpha$ increased over time, as it is the learning rate of the gradient of Q-function values ($R+\gamma\max_aQ(S',a)-Q(S,A)$), and, initially, they start off incredibly inaccurate, because they are initialized arbitrarily and over time converge to the true values, hence you'd want to weight them more as time increases rather than decrease it?
Why is this a convergence criterion?

Why is this a convergence criterion?

It is because $R$ and $S'$ are stochastic. A large learning rate applied when these values have variance would not converge to mean, but would wander around typically within some value proportional to $\alpha\sigma$ of the true value, where $\sigma$ is the standard deviation of the term $R + \gamma\text{max}_aQ(S',a)$. If you reduce $\alpha$ towards zero, then this expected error will also reduce to zero.
For deterministic environments, it should be possible to prove convergence with large $\alpha$.

In the special case of static policy, tabular learning and $\alpha = \frac{1}{N(s,a)}$ where $N(s,a)$ is number of visits to state $s$, action $a$, then the expected error for each Q value is the MSE from basic stats i.e. $\frac{\sigma_{TD}}{\sqrt{N(s,a)}}$


How does the CTC loss work? I am trying to implement CTC loss in Tensorflow, but their documentation is pretty limited. So I am not sure how to approach the problem. I found a good example in Theano: https://github.com/mohammadpz/CTC-Connectionist-Temporal-Classification/blob/master/ctc_cost.py.

Are any other resources that explain the CTC loss?

Also, I am trying to understand how its forward-backward algorithm works. And what is beam decoder in the case for CTC loss?

The paper Connectionist Temporal Classification: Labeling Unsegmented Sequence Data with Recurrent Neural Networks (2006), by Alex Graves et al., mentioned in the linked TensorFlow documentation, presents and explains the CTC loss and the CTC forward-backward algorithm (in section 4.1).

Have also a look at this article Sequence Modeling With CTC (2017) by Awni Hannun.


How can I perform lane detection with reinforcement learning? I'm quite new to reinforcement learning and my project will consist of detecting lanes with RL.

I'm using q-learning and I'm having a hard time thinking how my q table should look like, I mean - what could represent a state. My main idea is to feed the machine with a frame that contains a road picture, which the edge detection function is being applied to (and by thus getting lots of lines that exits in the frame). And train the machine which lines are the correct lane line. I already have a deterministic function that already recognizes the lanes and it will be the function that will teach the machine. I already organized some lane parameters such as (lane length, lane cords, lane color (white or yellow have a better probability to be a lane), lane diameter and the lane incline).

Now, my only issue is how should I construct the Q-table. Basically, what could represent a state and which lanes or decisions I should reward.

I will agree with malioboro, maybe RL is an overkill for such a task. Even though with the trend of autonomous driving research lately, papers dealing with lane changing almost certainly exist, you should check them out for more details.

As stated in Lane Change Decision-making through DeepReinforcement Learning with Rule-basedConstraints , "When the states are discrete and finite, the Q-function canbe easily formulated in a tabular form. But in many practical applications, for example, lane change decision-making task,the state space of them is very large or even continuous,using the Q-learning algorithm will lead to dimension disaster. Therefore, tabular Q-learning algorithm does not applicable to
the learning problem of continuous state space and continuousaction space"

So, i would recommend using image processing techniques, or Deep-Q-Learning. Q-Learning is not capable of dealing with continuous problems like lane changing or lane tracking.


Can a model, retrained on images classified previously by itself, increase its accuracy? Let's assume I have a CNN model trained to categorize some objects on the images. By using this model I find more categorized images. If I now retrain this model on data set that consists old set and newly categorised images is there a chance that such new model will have higher accuracy? Or maybe because new data posses only information that could be found on initial set, model will have similar/lower accuracy?

Please let me know if something unclear.

The most likely outcome of this approach is wasted time and very little effect on accuracy.

There will be changes to the model. Some will be beneficial and improve the model, but some will backfire making it worse.

For instance:

The model predicts with probability 0.4 that an image is in a certain class. It is the highest prediction, and actually true. It will be added to the training dataset with a "ground truth" of probability 1.0, so on balance more and better data has been added to the data set. This will improve generalisation, as whatever caused the relatively low 0.4 value initially - e.g. a pose or lighting variation - will now be covered correctly in the training set.
The model predicts with probability 0.4 that an image is in a certain class. It is the highest prediction, and actually false. It will be added to the dataset with "ground truth" of probability 1.0 for the wrong class. This will weaken associations to the correct class for similar input images, meaning for exaple that a certain pose or lighting difference that is already causing problems for the model will be used to incorrectly classify images in future.

These two scenarios will occur, on average, at a rate determined by the model's current test accuracy. So if your current model is 90% accurate, 1 in 10 images in your new training data will be mislabelled. This will  "lock in" the current errors at the same rate on average as they already occur.

The effect may be a drift up or down in accuracy as the model will definitely change due to the new training data. However you have little to no control over how this drift effect goes if you are not willing or able to oversee the automatic classifications generated on new data by the model.

There are a few ways to get some improvement unsupervised from new data. For instance:

Build an autoencoder from the early convolutional layers of your model and train it to re-generate all inputs as outputs. This should help it learn important features of the variations in data that you are using. Once this training is done, discard the decoder part of the auto-encoder and add your classifier back in to fine tune it. This may help if you have only a small amount of labelled data, but a lot of unlabelled data.
Use a model that has better accuracy than yours to auto-label the data. This might seem a little chicken-and-egg, but you may be able to create such a model using ensemble techniques. The ensemble model could be too awkward to use in production, but may still be used in an auto-labeling pipeline to improve your training data.

Note you may get even better results simply ignoring the extra unlabeled data, and instead fine tuning a high quality ImageNet-trained model on the labeled data you already have - saving yourself a lot of effort. Depends on the nature of the images, and how much labeled data you are already working with.


Could a Boltzmann machine store more patterns than a Hopfield net? This is from a closed beta for AI, with this question being posted by user number 47. All credit to them.

According to Wikipedia,

  Boltzmann machines can be seen as the stochastic, generative counterpart of Hopfield nets.

Both are recurrent neural networks that can be trained to learn of bit patterns. Then when presented with a partial pattern, the net will retrieve the full complete pattern.

Hopfield networks have been proven to have a capacity of 0.138 (e.g. approximately 138 bit vectors can be recalled from storage for every 1000 nodes, Hertz 1991).

As a Boltzmann machine is stochastic, my understanding is that it would not necessarily always show the same pattern when the energy difference between one stored pattern and another is similar. But because of this stochasticity, maybe it allows for denser pattern storage but without the guarantee that you'll always get the "closest" pattern in terms of energy difference. Would this be true? Or would a Hopfield net be able to store more patterns?

Indeed you're intuition is correct, a Boltzmann machine is able to hold more than a Hopfield network in its memory because of its stochastic nature as explored in this paper. In the paper they note that the capacity is around 0.6. After this ratio it starts to break down and adds much more noise to the recalled patterns.


What is the space complexity of breadth-first search? When using the breadth-first search algorithm, is the space complexity $O(b^d)$, where $b$ is the branching factor and $d$ the length of the optimal path (assuming that there is indeed one)?

The space complexity of the breadth-first search algorithm is $O(b^d$) in the worst case, and it corresponds to the largest possible number of nodes that may be stored in the frontier at once, where the frontier is the set of nodes (or states) that you are currently considering for expansion.
You can take a look at section 3.5 (page 74) of the book Artificial Intelligence: A Modern Approach (3rd edition, by Norvig and Russell) for more info about the time and space complexity of BFS.


Is there any artificially intelligent system that really mimics human intelligence? After having read something that Elon Musk said about artificial intelligence and how it could affect our lives, I've been reading about artificial intelligence, deep learning, etc. The recurrent topic is neural networks, which are used for "recognition tasks".
Although this is not my main question, can we say the following about neural networks?

Neural networks use the sigmoid function and gradient descent to fine-tune weights.

In any case, neural networks do not seem to really mimic any of the properties that we associate with intelligent beings, like humans, such as thinking outside the box, being able to observe, reflect and come up with innovations.
So, is there any artificially intelligent system that really mimics human intelligence?

As far as emulating an intelligent being, no. There are a few different potential architectures for possible AGI. Many of these are extremely infantile, as the bulk of AI research is in narrow AI, which focuses on creating algorithms that are highly specialized for a specific task.

With that being said, here is one supervised learning approach to this problem by Andrej Karpathy: Karpathy Method

OpenAI, also has a team who works on the AGI problem.


Programming an inference AI that computes the best outcomes like a quantum computer I bought an Intel Movidius Neural Compute stick a few weeks ago. Even though I can use it with the examples, I want to actually use it for something! The documentation is messy, and hard to work through.

I've already looked through some of the websites that have been mentioned in other posts.

I specifically want to make an inference/reward-based AI that tries to guess a bunch of numbers over a period of time. The AI has to guess the best 10 guesses, and then it gets shown the set. The closer answers are rewarded, and the farther answers are penalised.

In the end, I want it to try and guess the most likely set of numbers in a random number generator, kind of like how a quantum computer outputs answers.

This could be used for all sorts of things! Even if it's already built, I want to program it from scratch so I can learn how this sort of thing works (teach a man to fish type of thing).

So, considering my goals above, how should I start? Also, could this answer questions with close to the same accuracy as a quantum computer, if it's fed enough data?

Welcome to the AI stack Jake.

This probably isn't going to be possible. Modern Psudo-random number generators, like Mersenne Twister, are designed not to have any patterns in them, so there's nothing to learn from.

You could however, try something like predicting the values of a broken random number generator, like RANDU. These aren't used anymore, precisely because it's possibly to predict future numbers on the basis of past numbers.

Hope this helps.


Can we make a chatbot that UNDERSTANDS not just answer questions based on his database? Can we really make a chatbot that understands (rather than just replies to) questions based on the database/options of replies that it has? I mean, can it come up with correct/non-stupid replies/communications that don't exist in its database?

For example, can we make it understand the words but, if and so on? So whenever it gets a question/order it understands it based on "understanding". Like the movie "her" if you have watched it.

And all of this without using too much of code, just the basics to "wake it up" and let it learn from YouTube videos and Reddit comments and other data source like that.

This question has been studied academically for decades, and is really an extension of the work on Philosophy of Mind that was done in the two or three centuries before that.

A good resource is Mind Design II, though it's getting a little bit old now.

The modern schools of thought are:

Cognitivism. This is in decline, but was extremely popular in the 70's and 80's, and still fairly widespread in one form or another in the AI research community. It says that human brains really are just computers. If they're just computers, and that they're probably running a sort of symbolic reasoning algorithm like unification (although I think it's hard to find anyone who really thinks it's unification anymore). This is the idea underpinning work like SOAR. The main bottleneck, as Drayfus pointed out in the 1970's, is that you need to write down all the facts about something for a machine to "understand" it. "All the facts" turns out to rapidly turn into an infinite number for anything more complex than the smallest "microworlds" that you could deploy an AI program in. Searle also proposed his Chinese Room argument in response to this group, but it holds for Connectionist approaches as well (more on that later...).
Connectionism The connectionists hold that the complexity of our brains comes from massively parallel computation consisting of messages passed between billions of neurons in our heads. They think the correct approach to general AI is likely to involve simulations of similar architectures. It turns out that many of the things that are incredibly hard for Cognitivist projects (e.g. vision), are easy to solve with these approaches. The main criticisms from Cognitivists are that we don't have a very good idea of what these things are doing, and so the claim that they help us understand intelligence is false, and using them to solve practical problems might be dangerous. These are both somewhat fair, in my view. Older Cognitivist arguments, put forth most elegantly by Jerry Fodor, have now been discredited. Fodor argued that properties like language could never be understood as statistical artifacts of parallel computation, but he was wrong: all the best computational systems for language are now connectionist, and no one's ever made a cognitivist one that's even half as convincing. This is the dominant paradigm behind most of the modern advances in the field. Hinton's work forms the basis of most of the recent advances.
Dynamics Searle's argument was rooted in the idea that mapping inputs to outputs couldn't be what's happening in our heads, and that such a system couldn't be called intelligent. This also seems to be an implicit assumption in your question. The Dynamicists believe a variety of things, but I'd characterize them as collectively rejecting this idea. Authors like Paul Churchland argue that Searle's argument is rooted in a sort of pre-enlightenment "folk psychology". It's a bit like the theories that predated modern chemistry. Everyone was sure that fire was a substance that lived inside wood. If you heated up the wood properly, it could get out of the wood, making more heat. On the surface this seems pretty reasonable, but of course, it's wrong: The fire is actually a mixture of the wood with oxygen in the air, forming a new gas. There's no fire inside the wood. Similarly, Churchland would argue that there's no "Consciousness" inside us, allowing us to control our actions in the way we popularly imagine. Subjective experience is more likely to be "along for the ride", and entirely or mostly separate from the intelligent behaviors we observe. Some researchers think it could be described by a sub-system that maps observations of what the rest of the brain does into "stories" for the rest of the brain to receive as a sort of summary digest. Active research in this area tends to focus on things like the insect metaphor, and the interaction of the machine with its environment. It was fairly popular in the 1990's, but the phenominal success of connectionist approaches in the 2000's has led to its decline. Probably the best known experiments were the work of Rodney Brooks.


How to figure out which words have the same meaning in two different languages? Imagine two languages that have only these words:

Man = 1,
deer = 2,
eat = 3,
grass = 4

And you would form all sentences possible from these words:

Man eats deer.
Deer eats grass.
Man eats.
Deer eats.

German:

Mensch = 5,
Gras = 6,
isst = 7,
Hirsch = 8

Possible german sentences:

Mensch isst Hirsch.
Hirsch isst Gras.
Mensch isst.
Hirsch isst.

How would you write a program that would figure out which words have the same meaning in English and German?

It is possible.

All words get their meaning from the information in which sentences they can be used. The connection with other words defines their meaning.

We need to write a program that would recognize that a word is connected to other words in the same way in both languages. Then it would know those two words must have the same meaning.

If we take the word "deer" (2) it has this structure in English

1-3-2
2-3-4

In german (8):

5-6-8
8-6-7

We get the same structure (pattern) in both languages: both 8 and 2 lie in first and last position, and the middle word is the same in both languages, the other word is different in both languages. So we can conclude that 8=2 because both elements are connected with other elements the same way.

Maybe we just need to write a very good program for recognizing analogies and we will be on the right track to creating AI?

You are implying that such ideas are novel, and that such tools do not exist. But the idea is very popular, and there are numerous tools.

  We need to write a program that would recognize that a word is connected to other words in the same way in both language. Then it would know those two words must have the same meaning.

You are describing the essence of known natural language processing (NLP) tasks such as word alignment (link words in different languages that have the same meaning) and, of course, machine translation.

While learning a machine translation model, we actually do discover which words (or parts of words, or sequences of words) in different languages have the same meaning.

Here are some concepts I would recommend for further study of this subject:

Word alignment, an example for a well-known and popular tool would be fast_align
Word embeddings, word2vec is a widely used tool
Modern machine translation with sequence-to-sequence models, well-known tools are fairseq, or Sockeye


Why do non-linear activation functions that produce values larger than 1 or smaller than 0 work? Why do non-linear activation functions that produce values larger than 1 or smaller than 0 work?
My understanding is that neurons can only produce values between 0 and 1, and that this assumption can be used in things like cross-entropy. Are my assumptions just completely wrong?
Is there any reference that explains this?

Why wouldn't they work?
Each neuron's output is equal to a function over the sum of all its weights multiplied by their corresponding neurons. If that function is the Sigmoid function, then the output is squashed from $[0,1]$. If the entire layer uses a SoftMax function, then the output of all neurons is squashed from $[0,1]$ and their sum equals 1. In other others, they represent a set of probabilities, where you can then use cross-entropy to optimize their values (cross-entropy measures the difference between two probability distributions).
ReLU and ELU are simply other types of functions, whose output is not limited to the range $[0, 1]$. They are differentiable, like other activation functions, and so they can be used in any neural network.


Is the stride applied both in the horizontal and vertical directions in convolutional neural networks? In the convolutional layer for CNNs, when you specify the stride of a filter, typical notes show some examples of this but only for the horizontal panning. Is this same stride applied for the vertical direction too when you're done with the current row?
In other words, say our input volume is 7x7, and we apply a stride of 1 for a 3x3 filter. Is the output volume 5x5? (which would mean you applied the stride in both the horizontal and vertical panning).
Is it possible to apply a different stride for each direction?

Yes, in Keras this is simply implemented by using a tuple for the stride argument of a convolutional layer, with each element of the tuple corresponding to the stride of each dimension.


Does depth-first search always stop when it has found the leftmost solution? I'm a fresh learner of AI. I was told that depth-first search is not an optimal searching algorithm since "it finds the 'leftmost' solution, regardless of depth or cost". Therefore, does it mean that in practice, when we implement DFS, we should always have a checker to stop the search when it finds the first solution (also the leftmost one)?

One of the more standard assumptions when first introducing new students to search algorithms (like Depth-First Search, Breadth-First Search which you've also likely heard about or will hear about soon, etc.) is indeed that our goal is to find some sort of solution, and only find one.

If our intention is to find just a single solution, then yes, you will need to check at every node whether that is a solution node, and you can stop the search process once you've found one.

In practice, there can be all kinds of variants of this idea. Maybe in a different case you are interested in finding all solutions, rather than a single one; in such a case you would naturally not stop the search process after finding the first one, but continue searching.

So, to conclude, it really depends on exactly what you want, why are you using a search algorithm. If you only care about finding a solution, you can stop when you have one.


What to do with duplicate Innovations in a genome? I´m currently implementing NEAT.
What should I do when in a mutation the same Innovation occurs which has already happened to that genome?

Should I simply ignore it?

If not what do I do with it in the mating part?

Because this is more of an answer than a comment:

  In the add connection
  mutation, a single new connection gene with a random weight is added connecting
  two previously unconnected nodes.

So two nodes that have been mutated with a connection, can't remutate because the nodes should be unconnected.

  In the add node mutation, an existing connection is
  split and the new node placed where the old connection used to be. The old connection
  is disabled and two new connections are added to the genome.

The old connection is disabled, so the same mutation can't occur again on this connection. In practice, you can just remove the disabled connection, because it will be formed again with the add connection mutation.


Are information processing rules from Gestalt psychology still used in computer vision today? Decades ago there were and are books in machine vision, which by implementing various information processing rules from gestalt psychology, got impressive results with little code or special hardware in image identification and visual processing.

Are such methods being used or worked on today? Was any progress made on this? Or was this research program dropped? By today, I mean 2016, not 1995 or 2005.

Rules based on Gestalt psychology could be seen as a 'local minima' in terms of optimal image processing. Some of them could be surprisingly effective, but difficult to extend and improve upon, since they assume certain high level attributes that might not generalize well.
"If it's circular, then it's a fruit 90% of the time"
Modern methods like neural networks come at the problem from another direction, and try to build towards those high level attributes incrementally. You could say that a goal of these modern methods is to recreate a version of those 'assumptions' or 'Gestalt rules' empirically. That way, we can feel more confident that we're not using a locally optimal approach.
"This is the smallest algorithm that will accurately identify fruit 90% of the time"
Are such methods being used or worked on today?
Yes, and not just in image processing. If you look at any industry where fast processing of ambiguous domain-specific data is needed, there are usually systems that were designed by experts in that field using 'hand-crafted' heuristics. A benefit of these approaches is that the 'Gestalt rules' are understandable apart from the system that uses it, and is therefore easier to trust.
Was any progress made on this? Or was this research program dropped?
Aside from trivial cases, employing the gestalt approach in a particular domain requires expertise. "Image Expert" is an expensive and hard to fill role. Even then, you're bounded by that expert's ability to effectively generalize an extremely complex problem. Over the past decades, machine learning has become a lot easier and less expensive to employ, and has been slowly replacing all those 'hand-crafted' approaches.
However, one of the interesting things we're finding when we let the machine learning approaches loose is that they sometimes don't agree with our previous rules. Why they don't agree, and what this means about automation and our psychological biases, is one of the big open questions in AI.
You might also be interested in Saliency Maps, and similar techniques intended to visualize the general rules used by neural networks: https://en.wikipedia.org/wiki/Saliency_map
References:
http://wayback.archive-it.org/219/20121229060421/http://www.cs.indiana.edu/~jwmills/ANALOG.NOTEBOOK/klm/klm.html
https://journals.sagepub.com/doi/pdf/10.1177/002029400403701001
https://maritime.org/doc/op1140/index.htm
https://airandspace.si.edu/stories/editorial/inventing-apollo-spaceflight-biomedical-sensors


Why not use the target network in DQN as the predictor after training Target network in DQN is known to make the network more stable, and the loss is like "how good I'm now compared to using the target". What I don't understand is, if the target network is the stable one, why do we keep using/saving the first model as the predictor instead of the target?
I see in the code everywhere:

Model
Target model
Train model
Copy to target
Get loss between them

At the end, the model is saved and used for prediction and not the target.

The learning model and target model are only different by N steps (typically a few thousand) out the entire taining process. If the process is near complete, they will also be quite similar.
The target model is not inherently more stable in terms of producing "correct" or "better" Q values. Instead it is kept static for a period of time in order to stabilise the temporal difference (TD) target $R_{t+1} + \gamma \text{max}_{a'} \hat{q}(S_{t+1}, a', \theta^{-})$
Due to the copying stage that you listed:

If you return the target model, this is identical to returning the learning model from N steps beforehand.

Which in turn means that there was no point in doing the last N steps of training. You may as well have returned the learning model at the same point as copying.

Or another way of thinking about it: Returning the learning model at the end of the training process is identical to copying it to the target model as normal, then returning the target model immediately.


Machine learning algorithm for xml manipulation Given a virtual game map (picture) and a racing car at the map's starting point, I'm trying to build an algorithm that would help me generate a route that would get the car from the beginning to the endpoint.

route definition: a very complex .xml file that includes all the data the car needs in order to navigate the road successfully.

What I have:

I have thousands of different maps and I could get any number of maps I'd like.
For every map, I have a complex external algorithm using picture analysis that builds a route for me - although this route is not so accurate.
For some percentage of the maps, I also have a specific "good working car solution" - which is basically a manually built route file.

What I'm trying to achieve:
Basically, I'm trying to improve the complex route generator algorithm which I don't have access to.
I want to use the percentage of manual routes I have and compare them to the routes that are generated for the same maps. From the comparison and the differences between the manual and the automatically created route files, I want to build a second-step smart algorithm that "learns" what should be changed in the automatically generated routes and have that algorithm run as a second step and make it as close as possible to the handmade route.

My final goal is to be able to build an accurate route for maps I don't have the manual route for. I want to be able to produce an accurate route for every map - using both the external algorithm and the one I will build on top of it.

This entire subject is very new to me and I'd like to get your advice as to how should I approach this complex problem, and what could make this task easier for me.

Terminology

There are two uses of the word map in this discussion.

Road maps are construed below as images of road maps.
Mapping input to desired output is the skill the system must learn.

The set of examples used to teach the system from an existing mapping of input to output is called a labeled data set and the associated type of learning is called supervised.

Training Resources

Unreliable labeled road map examples and access to more also unreliable labeled road map examples
A large number of unlabeled road map examples and access to more road maps
A smaller number of labeled road map examples

Inputs

Virtual game map as an image
Starting position
Target ending position

Output

The fastest route in sufficient detail to fully direct car movement — Assuming that the route should be optimized for soonest arrival time, not for fuel conservation, minimal tire wear, safety, or minimal distance, since the car was identified as a race car.

Accurate Valuation of the Resource Inventory

The low quality routes from the black box service provide neither examples of desired system behavior nor examples undesirable system behavior.  The former case would be good training data.  The later would be good to use in an adversarial architecture based on the design of GANs and their variants.  The uncertain quality of the labeled data from the black box service makes it ambiguous and, from an information science point of view, of zero value.

Comparing the low quality routes and the high quality routes may be interesting, but not particularly useful given the current state of machine learning.  The objective of your initial project phase is to teach the machine to generate a high quality route from the low quality route, not produce a comparison report.  To do that without processing the images a substantially large overlap between these two sets would be required.

The smaller number of labeled road map examples for which the labels were created manually
Corresponding labeled road map examples for which the labels were created by the black box service

That approach would require you to correct bad routes to create a sufficient training set.  Without overlap, you have no training data for artificial network training.  It may be possible to create an algorithm of GAN style that learns how to correct the black box service output using a concept called cross-entropy, but it would require processing the images and would likely be more difficult to do so than to replace the black box service altogether.

If your ultimate goal is to create a working system that generates routes from images and start and and positions, I suggest discounting the external service altogether, and discard the idea of creating a route improver sub-system.  That you wish to improve upon the black box route generator's algorithm is noble but ultimately a time-consuming distraction to reaching the ultimate goal.

Replace the unreliable thing with the reliable thing you design and build and to which you will have full and open access to improve.  Your system will ultimately have to deal with the images in your code either way, and that's the most challenging aspect of the overall task.  Just learn about CNNs and RNNs and get right to it.  That's my advise anyway.

A Note on Feedback

For any non-trivial route, the car will not likely stay on the road if given only turn, acceleration, and deceleration instructions.  Detection of the edges of the road would normally be needed as a feedback mechanism to accompany the route instructions.  The only way around this is to make the visualization sufficiently accurate that cumulative errors never exceed the distance that would put the car off pavement.

A Note on Data Representation

JSON has a more flexible and terse structure when it comes to homogeneous arrays than XML, and it is easy to convert XML to JSON.  Furthermore, a transportation route is often represented as a directed graph, and there are many algorithms already conceived in graph theory that have been implemented in graph libraries in most languages.  For instance, shortest path, path equivalence, path concatenation, and detection of rings (driving in circles) are one line calls to these libraries.  Because JSON, for the reason given, has overtaken XML in many domains, the number of graph libraries that can read and write JSON directly has overtaken the number that can read or write XML directly.  The tooling for JSON analysis and visualization has surpassed that of XML at this point too.


YOLOv3 Synthetic Data Training Suppose we want to train a model to detect various objects.   Let's say we have training data of those objects in various backgrounds along with their bounding boxes. Basically these objects have been three dimensionally created and the bounding boxes have been drawn on them. Then these have been "synthetically inserted" into various blank backgrounds.

Why would a model trained only on this data do better than a model that has this data along with "real" data of these objects with their bounding boxes manually drawn?

I am not absolutely sure, but I guess it is due to the domain gap. As far as I have seen in my project where YOLOv3 was trained on synthetic images, it performed better when the model was trained and tested on the same domain (synthetic), while the performance drops when we introduce real images for testing. So when you include real images along with synthetic images, you might have to use some domain adaptation methods to improve the performance.


What are some conferences for publishing papers on graph convolutional networks? What are some conferences for publishing papers on graph convolutional networks?

Based on past publications, here are some journals and conferences where you can possibly publish or present a research paper on geometric deep learning or graph neural networks

Neural Information Processing Systems (NIPS)
International Conference on Learning Representations (ICLR)
Conference on Computer Vision and Pattern Recognition (CVPR)
International Conference on Machine Learning (ICML)

The website http://geometricdeeplearning.com/ also provides some information about the topic and links you to several workshops, papers, and tutorials.

Here are some links to some of the past workshops on GDL.

Representation Learning on Graphs and Manifolds (and the related homepage)
Workshop IV: Deep Geometric Learning of Big Data and Applications
Geometric Deep Learning on Graphs and Manifolds.


Regression using neural network I'd like to ask for any kind of assistance regarding the following problem:

I was given the following training data: 100 numbers, each one is a parameter, they together define a number X(also given).This is one instance,I have 20 000 instances for training.Next, I have 5000 lines given, each containing the 100 numbers as parameters.My task is to predict the number X for these 5000 instances.

I am stuck because I only know of the sigmoid activation function so far, and I assume it's not suitable for cases like this where the output values aren't either 0 or 1.

So my question is this : What's a good choice for an activation function and how does one go about implementing a neural network for a problem such as this?

Lets mock some data up.

  "100 numbers, each one is a parameter, they together define a number X(also given)"

# i.e. size of X_train -> [n x d]
# i.e. size of X_train -> [??? x 100]  , when d = 100

# "I have 20000 instances for training"
# i.e. size of X_train -> [20000 x 100], when n = 20000

import torch
import numpy as np

X_train = torch.rand((20000, 100))
X_train = np.random.rand(20000, 100) # Or using numpy

But what is your Y?

# Since the definition of a regression task,
# loosely means to predict an output real number
# given an input of d dimension

# So the appropriate Y_train would
# be of dimension [n x 1]
# and look like this:

y_train = torch.rand((20000, 1))

y_train = np.random.rand(20000, 1) # Or using numpy

What is a linear perceptron?

Taking definition from this tutorial

Thus, in picture:

Next we need to define training routine,

For now take it as biblical truth that this is an okay routine to train a neural net model (this isn't the only way but easiest or supervised learning):

In code:

import math
import numpy as np
np.random.seed(0)

def sigmoid(x): # Returns values that sums to one.
    return 1 / (1 + np.exp(-x))

def sigmoid_derivative(sx):
    # See https://math.stackexchange.com/a/1225116
    # Hint: let sx = sigmoid(x)
    return sx * (1 - sx)

def cost(predicted, truth):
    return np.abs(truth - predicted)

num_epochs = 10000 # No. of times to iterate.
learning_rate = 0.03 # How large a step to take per iteration.

# Lets standardize and call our inputs X and outputs Y
X = np.array(torch.rand((20000, 100)))
Y = or_output

for _ in range(num_epochs):
    layer0 = X

    # Step 2a: Multiply the weights vector with the inputs, sum the products, i.e. s
    # Step 2b: Put the sum through the sigmoid, i.e. f()
    # Inside the perceptron, Step 2.
    layer1 = sigmoid(np.dot(X, W))

    # Back propagation.
    # Step 3a: Compute the errors, i.e. difference between expected output and predictions
    # How much did we miss?
    layer1_error = cost(layer1, Y)

    # Step 3b: Multiply the error with the derivatives to get the delta
    # multiply how much we missed by the slope of the sigmoid at the values in layer1
    layer1_delta = layer1_error * sigmoid_derivative(layer1)

    # Step 3c: Multiply the delta vector with the inputs, sum the product (use np.dot)
    # Step 4: Multiply the learning rate with the output of Step 3c.
    W +=  learning_rate * np.dot(layer0.T, layer1_delta)

Now that we learn the model, i.e. the W.

When we see the data points that we need to use the model on, we apply the same forward propagation step, i.e. layer1 = sigmoid(np.dot(X, W))

Since we have:

  I have 5000 lines given, each containing the 100 numbers as parameters.My task is to predict the number X for these 5000 instances.

And in code:

# If we mock up the data,
# it should be the same internal dimension.
X_test = np.random.rand(5000, 100)

# The desired output just needs to pass through the W and the activation:
# the shape of `output` -> [5000 x 1] ,
# where there's 1 output value for each input.
output = sigmoid(np.dot(X_test, W))


Is there any paper, article or book that analyzes the feasibility of acheiving AGI through brain-simulation? In my understanding, the mind arises from a physical system, the brain. I see that there is a big research under the topic of simulating physical systems efficiently (especially in quantum computing). Hence, in theory, we could achieve AGI by simulating the physical brain.

Is there any research I should look into regarding this topic? I would like to hear if it is possible, why, why not, what are the limitaions, how far we are from achieving this, and anything else, really. Also, I would like to read something about my first assumption ("the mind arises from a physical system, the brain").

I have searched AI.SE but I've found only related questions, so I don't think this is duplicate. For reference:

Why are we asking, “How can we simulate the brain?”
How powerful a computer is required to simulate the human brain?
Are there any artificial neuromorphic systems which can mimic the
brain?
Is it possible to build human-brain-level artificial intelligence based on neuromorphic chips and neural networks? (this one is pretty similar but I think it's slighly different, and it doesn't answer my question).

Note: I am not asking for the possibility NOW, but in general, so telling me that "we don't know the brain enough" is not on topic.

The Title Question

Is there any paper, article or book that analyzes the feasibility of achieving AGI through brain-simulation?

Yes. There are various analyses that have been published. We have some early work like Some Philosophical Problems From the Standpoint of Artificial Intelligence, John McCarthy and Patrick J. Hayes, Stanford University, 1969. And there is more recent and optimistic work like Essentials of General Intelligence: The Direct Path to Artificial General Intelligence, Peter Voss, 2007. There are refutations of the concept of General Intelligence, artificial or not, in some posts here (listed below) and quite a bit of academic scrutiny of the assumptions behind the AGI concept and the offspring of it: The Singularity. Concern about The Singularity is a social phenomenon, not a scientific one. It began with the depiction of the emergence of a super-intelligence as an inevitability in science fiction and in sensational media.
But mathematically terse proofs of AGI feasibility are not found in the media or in the Terminator franchise started by the wonderful imagination of James Cameron. This is not to say that world domination cannot fall into the hands of a new species created by humans. In fact, Jaques Ellul, in his Techological Society points out that in many ways humans are already subservient to technology — that the creation of humankind is already autonomous and dominant. His 449 page heap of evidence is witty and quite convincing.
Academic publications either

(a) Assume feasibility and discuss approaches to its design or

(b) Illuminate caveats in the idea of a universally intelligent system

Where is the Mind?
Whether the entirety of the mind arises from a physical system is a question that dates back to René Descartes and Gottfried Leibniz. Current conceptions of intelligence tend to deny the interaction and interdependence of the human brain, the rest of the human, and the biosphere into which the human fits without extreme expenditure to produce a floating, underwater, or underground simulation of the biosphere. The mind may center in the brain, but any brain disconnected from the biosphere and the circulatory system for even a short time is completely useless. A think tank is not a bunch of smart people in a sensory deprivation tank. Quite the opposite. Early experimentation with think tanks revealed the importance of mental health of the participants through diversion and physical exercise so that the various forms of neurological stasis could be maintained.
It is quite questionable whether computers without bodies will ever learn as much of general use (that can be put to use in the biosphere) without moving about and interaction with the biosphere. It is highly likely that the mind is more like Carl Jung sees it, as in the brain and elsewhere. Ludwik Fleck, in a somewhat Jungian fashion, speaks of science as the product of a collective in his Genesis and Development of a Scientific Fact, a good read that throws considerable light on the folly of most people's naive conception of facts.
Quantum Computing Buzz
When Intel's founder, Robert Noyce, brought in Gordon Moore in 1975, it was because of various statistical observations he had made, including what is now known as Moore's Law, a curve that on semi-log paper in 1965 looked linear. Since then, the business assumption in Silicon Valley has been that compactness and speed would double frequently and consistently. In the last few years, speed has stopped doubling and compactness seems to have reached its limit around 7 nm. The transistors in this VLSI technology have only a handful of atoms to represent the semiconductor topologies that permit gating, the fundamental activity of digital circuitry.
In essence, to continue the Silicon Valley boon, it has to become Particle Valley, using carbon allotropes or other nano-tech to build computers where Moore's Law is not overturned by the limitations of the structure of the universe. Whether this can be done requires its own feasibility study. Already, SSD (solid state drive) technology relies heavily on probability. Errors are occurring in great proportions and error correcting techniques are used to keep them manageable. That's because, as we reach for designs closer to the quantum level, Brownian motion or the light electromagnetic interference can upset the digital operation intended.
Challenges Versus Show Stoppers
As with many trends in the history of human endeavors, determining what is a challenge and what is a show stopper is not clear until viewing in hindsight. That is why people are asking the various questions in this Stack Exchange listed in the question.
Stating that mentioning our partial knowledge of brain operation is off topic for this question might have been appropriate if we were trying to invent something that we had not already seen in operation in nature. However, that is not the case. We call it artificial intelligence because we think that we are intelligent and want to extend that facility we find in ourselves. The following fields, in addition to quantum physics and machine learning, are thus necessarily relevant to the feasibility of a more generally usable fabricated intelligent system.

Neuro-chemistry in general
Genetics as it applies to neurological network structuring and cell activity
Correlation between DNA coding and human mental performance and stability
Applied psychology
Addiction science
Cognitive science
Mathematics in general and probability and statistics specifically
Control theory
Philosophy, especially in regard to whether intelligence actually exists and the ideas of culturally determined knowledge and belief
Language and linguistics
Symbology
Conceptions of a thought collective

Not only do we not know enough in these areas, but we know very little. There is also considerable evidence that humanity tends to forget some of the things it once knew. Take away our petroleum and we may find out just how much we forgot.
Whether any of the things we have lost from our general knowledge are prerequisites to solving the challenges of universally intelligent system design is unknown. A simple example is that many computing science graduates do not know anything about Kurt Gödel's incompletness theorems and how they led to Alan Turing's completeness theorem. Many use back-propagation in their Python code but don't have any conception that it is a corrective feedback distribution scheme.
On the flip side, we have no strong argument to draw the conclusion that there is some theoretical limitation as solid as the second law of thermodynamics, one of the apparently insurmountable hard stops built into our physical reality on and off planet. We have no proof that intelligence does not exist and is rather an anthrocentric fantasy. We have no proof that whatever limitations humans have cannot be overcome by copying the intellectual life of human beings into human creations or using intellect to create greater intellect in machines.
What we know, as smart as we think we are, is greatly outweighed by what we don't know. Anyone in real research work will agree.

Refutations of AGI Philosophy
Some of the many refutations of AGI that appear throughout academia. Some articles include these.

What is AGI?, Luke Muehlhauser, 2013

Philosophy will be the key that unlocks artificial intelligence, David Deutsch, 2012, The Guardian

On Artificial Intelligence, Rodney Brooks, 2018, The Library of Economics and Liberty

These are some posts here that question various assumptions prevalent in popular AI media.

Who was the first person to recognize the distinction between human-like general intelligence and domain-specific intelligence?


Can you use machine learning for binary data? I am totally new to artificial intelligence and neural networks and have a broad question that I hope is appropriate to ask here.
I am an ecologist working in animal movement and I want to use AI to apply to my field. This will be one of the few times this has been attempted so there is not much literature to help me here.
My dataset is binary. In short I have presence(1) and absence(0) of animal locations that are associated with a series of covariates (~20 environmental conditions such as temperature etc.). I have ~1 million rows of data to train the model on with a ratio of 1:100 (presence:absence).
Once trained, I would like a model that can predict if an animal will be in a location (or give a probability) based on new covariates (environmental conditions).
Is this sort of thing possible using AI?
If so where should I be looking for resources. I write in 'R', should I learn 'Python'?
Any input is much appreciated here!

Of course you can use AI (specially Deep Learning) in your application. your covariates will be the input to your AI model and the model should predict probability of presence. The model has no problem with binary data and binary data is common in this field.
Also note that 1:100 ratio is not good and the network will probably learn to output absence for any input (this way it gets 99% accuracy but really it's not doing anything). So you should probably balance them (using almost same data, or telling network to pay more attention to presence data (by weighting the related loss)).
I think nowdays you can find Deep Learning in any popular coding language. But most of DL community use python and its really easy to learn.
If you want to learn Deep Learning there are a lot of sources on internet. But I suggest you the Deep Learning courses of Deeplearning.ai in coursera (If you have a lot of time) and CS231n of Stanford university on youtube (If you have time)


Can reinforcement learning be used for tasks where only one final reward is received? Is reinforcement learning problem adaptable to the setting when there is only one - final - reward. I am aware of problems with sparse and delayed rewards, but what about only one reward and a quite long path?

RL can be used for cases where you have sparse rewards (i.e. at almost every step all rewards are zero), but, in such a setting, the experience the agent receives during the trajectory does not provide much information regarding the quality of the actions.
Games can be often formulated as episodic tasks. For example, you could formulate a chess match as an episode and you could give a (non-zero) reward only at the end of the match. However, in this specific case, it will be hard for the RL to understand which moves have mainly contributed to the reward received, which is known as the credit assignment problem.
You can solve the issue of sparse rewards with reward shaping (in particular, potential-reward shaping).
The term "delayed rewards" may also refer to the cases where you receive only one reward at the end of the episode, although it may more usually refer to scenarios where the reward at one-time step is only received later (for some reason).


Neural Network with varying inputs (for a game ai) I want to create a simple game which basically consists of 2d circles shooting smaller circles at each other (to make hitbox detection easier for the start). My goal is to create an ai which adapts its own behaviour to the player‘s. For that, i want to use a NN as brain. Every frame, the NN is fed with the same inputs as the player and his output is compared to the players output. (outputs in this case are pressed keys like the up-arrow)
As inputs, I want to use a couple different important factors:
for example, the direction of the enemy player as number from 0 to 1

I also want to input the direction, size and speed of enemy’s and own projectiles and this is where my problem lies. If there was only one bullet per player, it would be easy but I want the number of bullets to be variable so the number of input neurons would have to be variable.

My approaches:
1) use a big amount of neurons and set unused ones to 0
( not elegant at all)
2) Instead of specific values, just use all the pixels‘ rgb values as inputs (would limit the game as colours would deliver all the information) (+factors like speed and direction would probably not have any impact)

Is there a more promising approach to this problem ?
I hope you can give me some inspiration.

Also, is there a difference in ranging input values between 0/1 or -1/1 ?

Thank you in advance, Mo

Edit: In case there aren‘t enough questions for you, is there a way to make the NN remember things ? For example, if I added a mechanic to the game which involves holding a key, I would add an input neuron which inputs 1 if the certain key is pressed and 0 if it isn‘t but I doubt that would work.

The most generic approach is to input all pixels as you have suggested. A CNN would be the best architecture for that. To provide information like speed or velocity, you can feed more than one frame to the CNN (e.g. the last 5 frames or whatever provides enough information). The CNN can learn movement information by comparing those images.

If you want to store additional information (like an inventory item), an input neuron for each value would be an option. You can also look up LSTM (long-short term memory) models, but for your specific situation a hardcoded neuron would be the easier solution.


Why are there two versions of softmax cross entropy? Which one to use in what situation? I have seen 2 forms of softmax cross-entropy loss and are confused by the two. Which one is the right one?
For example in this Quora answer, there are 2 answers:

$L(\mathbf{w})=\frac{1}{N} \sum_{n=1}^{N} H\left(p_{n}, q_{n}\right)=-\frac{1}{N} \sum_{n=1}^{N}\left[y_{n} \log \hat{y}_{n}+\left(1-y_{n}\right) \log \left(1-\hat{y}_{n}\right)\right]$

$\mathrm{L}(y, \hat{y})=-\Sigma y(i) \log \hat{y}(i)$, which is only the first part of the version one.

It's the same thing, first version is the special case of the more general one. In the first case you only have two classes, it's binary cross-entropy, and they also included iteration over batch of samples. In the second case you have multiple classes and in the current form it's only for a single sample.
In the first case there is only one output, if you had two outputs it would have been
\begin{equation}
-\frac{1}{N} \sum_{n=1}^N \sum_{j=1}^2 y_{n,j} \log(\hat y_{n,j})
\end{equation}
where $n$ iterates over batch samples, and $j$ over two classes. The reason why it was written like that is that if you have two classes you can have only one output because you can immediately conclude about probability of the second class if you have the probability of the first class, it would simply be $p_1 = 1-p_0$.
In the second case, with batch samples included, it would be
\begin{equation}
 -\frac{1}{N} \sum_{n=1}^N \sum_{j=1}^c y_{n,j} \log(\hat y_{n,j})
\end{equation}
wheren $n$ iterates over batch samples, and $j$ over $c$ output classes.


Is it possible that the fittest individuals in an Artificial Life population may be successful by not actively pursuing the rules of the environment? I'm trying to understand Artificial Life (e.g. here for a simple background) in Computational Evolution.
I understand that in this set of methods, you set up a dynamic environment (e.g. the ecology of the environment) and then you set a series of rules; e.g.:

You need energy to reproduce.
You intake energy from food sources.
For nourishment, you can
eat plants, animals, or steal food.
You must stay alive until you reproduce.
Every action consumes energy.
When you have no energy left, you die.

I think I need a set of rules that govern the survival of an artificial life. You run the environment and see what persists (there's a set of rules instead of a fitness score), and the individuals that survive are said to be successful.
I can imagine a scenario where a successful organism in this environment consumes a lot of food, reproduces, but possibly runs out of energy and dies. I'm wondering if there's ever a situation where an organism does very little (or nothing), and still be successful? I'm not sure if this question makes sense, please let me know if clarification is needed. Given the specified environment, I want to know if the most active organism will always be the most successful. The most active organism would be the one that obtains the most food/energy/reproduces the most. Or is it possible to not be the most active organism and still be successful?

It is entirely possible!
You see, the agents will perform whatever actions are available to them, and if the evolutionary algorithm is setup correctly, whatever set of actions provides them with a bigger survival rate will be the one that gets explored and reproduced the most.
Here is a very interesting list of "Specification Gaming" in AI, where the agents happened to "game" the rules to reach their goals (metric optimization) without actually doing what the creators intended: https://docs.google.com/spreadsheets/u/1/d/e/2PACX-1vRPiprOaC3HsCf5Tuum8bRfzYUiKLRqJmbOoC-32JorNdfyTiRRsR7Ea5eWtvsWzuxo8bjOxCG84dAg/pubhtml (second link)


Why studying machine learning is an opportunity in today's world? I just wanted to gather some perspective on why this is a great opportunity to be able to study machine learning today?

With all the online resources (online courses like Andrew Ng's, availability of datasets such as Kaggle, etc), learning machine learning has become possible.

I understood that you can have high paid jobs; but you also need a lot of work dedication to be good at it, which makes your salary not so attractive! (in comparison to the number of hours you spend to keep up with this fast moving field)

Why it is so desirable to take this opportunity and start learning machine learning today? (community, ability to start a business, etc.)

What sort of opportunity it is depends on how much you want to focus on it.

If you want to be a regular programmer, you might take the time to learn a high level interface for some machine learning tools, such as Tensorflow or Keras. There will be plenty of things you don't know how to do (even within those tools), but you may be able to apply predesigned model architectures to problems. The models won't be as good as one designed specifically for the problem, but it's one more tool in your toolbox, and it's possible you'd be able to get some useful results occasionally without devoting a huge amount of time to mastering the techniques.

But if you want to really focus on machine learning, at the research level, you can potentially tackle problems that existing techniques haven't been able to solve. This is where most of the big projects that you've probably heard of will be happening: self-driving cars, AlphaGo, etc. What you can expect here is a lot of hard work. You will need to develop a fairly deep understanding of the mathematics involved so you can visualize (to some extent) what is happening in the potentially high dimensional spaces involved, identify potential failure modes, and identify models that won't fall into them. It involves a lot of trial and error, failed attempts and gradual improvement before you're able to develop a model that beats the stuff already out there.

It's very rewarding work if you enjoy it. There are well-paying positions in the field, but that's just a bonus if you already enjoy the work, and it isn't enough of a bonus if you don't. In my opinion, going into this for just the money would be a mistake. There are almost definitely other jobs that pay just as well but don't take anywhere near the time investment to become (and stay) competent at them. But if you really want to work in this field for its own sake, and also want to make sure you don't starve while you're doing it, it's absolutely worth it.


What are some tactics for recognizing artificially made media? With the growing ability to cheaply create fake pictures, fake soundbites, and fake video there becomes an increasing problem with recognizing what is real and what isn't. Even now we see a number of examples of applications that create fake media for little cost (see Deepfake, FaceApp, etc.).

Obviously, if these applications are used in the wrong way they could be used to tarnish another person's image. Deepfake could be used to make a person look unfaithful to their partner. Another application could be used to make it seem like a politician said something controversial.

What are some techniques that can be used to recognize and protect against artificially made media?

Digital Media Forensics (DMF) field aims to develop technologies for the automated assessment of the integrity of an image or video, so DMF is the field you are looking for. There are several approaches in DMF: for example, those based on machine learning (ML) techniques, in particular, convolutional neural networks (CNNs).

For example, in the paper Deepfake Video Detection Using Recurrent Neural Networks (2018), David Güera and Edward J. Delp propose a two-stage analysis composed of a CNN to extract features at the frame level followed by a temporally-aware RNN to capture temporal inconsistencies between frames introduced by the deepfake tool. More specifically, they use a convolutional LSTM architecture (CNN combined with an LSTM), which is trained end-to-end, so that the CNN learns the features in the videos, which are passed to the RNN, which attempts to predict the likelihood of those features belonging to a fake video or not. Section 3 explains the creation of deepfake videos, which leads to inconsistencies between video frames (which are exploited in the proposed method) because of the use of images with different viewing and illumination conditions.

Other similar works have been proposed. See this curated list https://github.com/aerophile/awesome-deepfakes for more related papers.


Reduce receptive field size of CNN while keeping its capacity? I have a convolutional encoder (a CNN) consisting of DenseBlocks and a total of 50 layers (cf. FC-DenseNet103). The receptive field of the encoder (after last layer) is 660 according to Tensorflow function compute_receptive_field_from_graph_def(..)) whereas the input image is 64x64 pixels. Obviously the receptive field is way too big.

How can the receptive field be reduced to say 46 but the capacity of the encoder be more or less kept at the same level? By capacity I simply mean the number of parameters of the model. The capacity requirement is justified due to the complex dataset to be processed.

Using less layers or smaller kernels reduces the receptive field size but also the capacity. Should I then just increase the number of filters in the remaining layers in order to keep the capacity?

One way to keep the capacity while reducing the receptive field size is to add 1x1 conv layers instead of 3x3 (I did so within the DenseBlocks, there the first layer is a 3x3 conv and now followed by 4 times a 1x1 conv layer instead of the original 3x3 convs (which increase the receptive field)). In doing that, the number of parameters can be kept at a similar level. While 1x1 convolutions are good to add non-linearity, they are not useful to learn spatial information where neighboring pixels/values correlate. This might turn out to be a problem.

Therefore, experiments will have to show if this idea of adding 1x1 conv layers to keep the capacity proves useful or not in learning spatial features on a natural image data set.


If the accuracy of my current model is low ($50 \%$) and we want to minimize time in collecting more data, should we try other models? Suppose we have a data set with $4,000$ labeled examples. The outcome variable is trinary (three possible categorical values). Suppose the accuracy of a given model is "bad" (e.g. less than $50 \%$).

Question. Should you try different traditional machine learning models (e.g. multinomial logistic regression, random forests, XGBoost,
etc.), get more data, or try various deep learning models like
convolutional neural networks or recurrent neural networks?

If the purpose is to minimize time and effort in collecting training data, would deep learning models be a viable option over traditional machine learning models in this case?

To know if your model needs more training data, try to plot out "learning curves", that are based on increasing size of the training set.

Basically, you calculate training and validation accuracy metrics for 1, 2, 3, 4, 5, ..., m  training samples. Size of validation set may be constant over time. If the accuracy is still rising when your data set is fully used, then you need more training data.


How to represent a large decision tree? A system makes a decision basing on a large number of varied factors, following a "live" decision tree - one that is (independently, through other subsystem) updated with new decisions, new situations.

The individual decisions can be recorded as a kind of structure:

decision function
node to activate if decision is positive
node to activate if decision is negative

and a node can be another decision record, or a conclusion.

This isn't entirely a binary tree, as many decisions may lead to the same conclusion - each node has two children, but may have many parents.

There is absolutely no problem storing the tree in memory - it can be database records or entries of a map, or just a list. It's perfectly sufficient for the machine.

The problem here is building the subsystem that expands the decision tree - and in particular, having a human operator understand the structure being built, to be able to tune, guide, fix, adjust it: debugging the AI learning process.

The question is: how to represent that data in a human-readable way, that emphasizes the flow of the graph?

a non-working example of the answer is Concept map - in this case it only goes so far; with more than thirty or so nodes, it becomes a jumbled mess, especially if the number of cross-connections (multiple parents) becomes significant. Maybe there exists some way of laying it out or slicing it to make it clearer...?

The question is: how to represent that data in a human-readable way, that emphasizes the flow of the graph?

Train a reasoning engine to understand the decision tree for you.

Observe how IBM Watson/The Debater can

Receive a particular question
Find and read Wikipedia articles related to the question
Understand parts of those articles and generate human-relevant arguments for you.

Follow these steps:

Develop your decision tree however you normally would.
Train a reasoning engine that can output natural language about concepts within decision trees.
Apply reason engine from step one to decision tree in step one; repeat.


Does MMD-VAE solve the problem of blurred images of vanilla VAEs? I understand that with vanilla VAEs, there are a few reasons justifying the production of blurred out images. The InfoVAE paper describes the case when the decoder is flexible enough to ignore the latent attributes and generate an averaged out image that best reduces the reconstruction loss. Thus the blurred image.

How much of the problem of blurring is really mitigated by the MMD formulation in practical experiments? If someone has experience working with MMD-VAEs, I'd like to know their opinion on what the reconstruction quality of MMD-VAEs is really like.

Also, does the replacement of the MSE reconstruction loss metric by other perceptual similarity metrics improve generated image quality?

[Answering my own question after 5 months of studying VAE models]

The point of the MMD-VAE or InfoVAE is not exactly to emphasise on the visual quality of generated samples. It is to preserve greater amount of information through the encoding process. The MMD formulation stems from introducing a mutual coefficient factor into the Evidence Lower BOund (ELBO) loss of VAEs. Refer to the paper appendices for full derivation. This formulation improves information content in latent space and provides for more accurate approximation of the true posterior - these results have also been empirically proven in the paper.

However, the InfoVAE uses pixel-wise or element-wise reconstruction loss. An element-wise reconstruction loss is likely to lead to some extent of blurriness inrespective of the prior loss term. On Github, several developers have implemented the InfoVAE model and shown their results. Here is a link to one such implementation whose results I could personally verify.

From my own experimentations, I can say that even though InfoVAE may give better reconstructions for some data, there is still considerable blurriness.

Perceptual similarity metrics may be learned or computed as a static function of the input image. With a learned perceptual loss, VAEs can produce much sharper images. PixelVAE and VAEGAN are well-known models with such implementations. For a static function of the image itself, reconstruction quality will depend on the nature of that function and such a model may not be very useful for all kinds of datasets. Using measures like SSIM, FSIM, we may still end up getting blurred images.


How Graph Convolutional Neural Networks forward propagate? In the basic variant of GCN we have the following:

Here we aggregate the information from the adjacent node and pass it to a neural network, then transform our own information and add them all.
But the main question is: how can we ensure that $W_{k}(\sum(\frac{h_k}{N(V)})$ will be the same size as $B_{k}h_{v}$ and does $B_{k}$ emply another neural network?

I think the picture you're presenting is mostly for educational purposes and that's why they are excluding the node itself from it's neighbors and using two distinct networks (most of the papers I've read they are using the same network for the neighbors and for the center node). But you are right the two networks needs to have the same input and output shapes otherwise the point-wise summation between the two terms is not possible.


Why the number of training points to densely cover the space grows exponentially with the dimension? In this lecture (minute 42), the professor says that the number of training examples we need to densely cover the space of training vectors grows exponentially with the dimension of the space. So we need $4^2=16$ training data points if we're working on $2D$ space. I'd like to ask why this is true and how is it proved/achieved? The professor was talking before about K-Nearest Neighbors and he was using $L^{1}$ and $L^{2}$ metrics. I don't think these metrics induce a topology that makes a discrete set of points dense in the ambient space.

First, let's try to build some intuition for what we mean when we say that we want to "densely cover" a $d$-dimensional space $\mathbb{R}^d$ of real numbers. For simplicity, let's assume that all values in all dimensions are restricted to lie in $[0, 1]$. Even with just a single dimension $d=1$, there are actually already infinitely many different possible values even in such a restricted $[0, 1]$ range.
But generally we don't actually care about literally covering every single possible value. Generally, we expect that points in this $d$-dimensional space that are "close" to each other also "behave" similarly, that there's some level of "continuity". Hence, to get "sufficient" or "good" or "dense" coverage of the space, you can somewhat informally assume that every data point you have occupies some space around it. This is the intuition behind Lutz Lehmann's comment under your question: you can think of every point as being a $d$-dimensional cube occupying some volume of your $d$-dimensional space.
Now, if you have a $d$-dimensional space of size $[0, 1]$ along every dimension, and you have little cubes that occupy a part of that space (for instance, cubes of size $0.1$ in every dimension), you will indeed find that the number of cubes you need to fill up your space scales exponentially in $d$. The basic idea is: if some number of cubes $K$ is sufficient to fill up the $d$-dimensional space, and if you then increase the dimensionality to $d+1$, you'll need $dK$ cubes to fill the new space. When you add a new dimension, the complete previous space becomes essentially just one "slice" of the new space.
For dimensions $d = 1, 2, 3$, this is fairly easy to visualise. If you have $d=1$, your space is really just a line, or a line segment if you constrain the values to lie in $[0, 1]$. If you have a $[0, 1]$ line segment, and you have little cubes of length $0.1$, you'll need just ten of them to fill up the line.
Now imagine that you add the second dimension. Suddenly your line becomes an entire plane, or a $10\times10$ square grid. The $10$ cubes are now only sufficient to fill up a single row, and you'll have to repeat this $10$ times over to fill up the entire $2$D space; you need $10^2 = 100$ cubes.
Now imagine that you add the third dimension. What used to be a plane gets "pulled out" into an entire three-dimensional cube -- a large cube, which will require many little cubes to fill! The plane that we had previously is again just a flat slice in this larger $3$D space, and the entire strategy for filling up a plane will have to be repeated $10$ times over to fill up $10$ such slices of the $3$D space; this now requires $10^3 = 1000$ cubes.
Past $3$ dimensions, the story continues in exactly the same way, but is a bit harder for us humans to visualise.


How can I implement back-propagation for medium-sized neural networks? I've been wanting to make my own Neural Network in Python, in order to better understand how it works. I've been following this series of videos as a sort of guide, but it seems the backpropagation will get much more difficult when you use a larger network, which I plan to do. He doesn't really explain how to scale it to larger ones.
Currently, my network feeds forward, but I don't have much of an idea of where to start with backpropagation. My code is posted below, to show you where I'm currently at (I'm not asking for coding help, just for some pointers to good sources, and I figure knowing where I'm currently at might help):
import numpy

class NN:
    prediction = []
    def __init__(self,input_length):
        self.layers = []
        self.input_length = input_length
    def addLayer(self, layer):
        self.layers.append(layer)
        if len(self.layers) >1:
            self.layers[len(self.layers)-1].setWeights(len(self.layers[len(self.layers)-2].neurons))
        else:
            self.layers[0].setWeights(self.input_length)
    def feedForward(self, inputs):
        _inputs = inputs
        for i in range(len(self.layers)):
            self.layers[i].process(_inputs)
            _inputs = self.layers[i].output
        self.prediction = _inputs

    def calculateErr(self, target):
        out = []
        for i in range(0,len(self.prediction)):
            out.append(  (self.prediction[i] - target[i]) ** 2  )
        return out

class Layer:

    neurons = []
    weights = []
    biases = []
    output = []

    def __init__(self,length,function):
        for i in range(0,length):
            self.neurons.append(Neuron(function))
            self.biases.append(numpy.random.randn())

    def setWeights(self, inlength):
        for i in range(0,inlength):
            self.weights.append([])
            for j in range(0, inlength):
                self.weights[i].append(numpy.random.randn())

    def process(self,inputs):
        for i in range(0, len(self.neurons)):
            self.output.append(self.neurons[i].run(inputs,self.weights[i], self.biases[i]))

class Neuron:
    output = 0
    def __init__(self, function):
        self.function = function
    def run(self, inputs, weights, bias):
        self.output = self.function(inputs,weights,bias)
        return self.output

def sigmoid(n):
    return 1/(1+numpy.exp(n))

def inputlayer_func(inputs,weights,bias):
    return inputs

def l2_func(inputs,weights,bias):
    out = 0

    for i in range(0,len(inputs)):
        out += weights[i] * inputs[i]
    out += bias

    return sigmoid(out)

NNet = NN(2)

l2 = Layer(1,l2_func)

NNet.addLayer(l2)
NNet.feedForward([2.0,1.0])
print(NNet.prediction)

So, is there any resource that explains how to implement the back-propagation algorithm step-by-step?

Backpropagation isn't too much more complicated, but understanding it well will require a bit of mathematics.
This tutorial is my go-to resource when students want more detail, because it includes fully worked through examples.
Chapter 18 of Russell & Norvig's book includes pseudocode for this algorithm, as well as a derivation, but without good examples.


Is there a machine learning algorithm to find similar sales patterns? I have a dataset as follows

(and the table extends to include an extra 146 columns for companies 4-149)

Is there an algorithm I could use effectively to find similar patterns in sales from the other companies when compared to my company?

I thought of using k-means clustering, but as I'm dealing with 150 companies here it would likely become a bit of a mess, and I don't think linear regression would work here.

I would recommend a hierarchical cluster algorithm, after normalising your numbers into proportions. Then the clustering should be able to identify similar patterns. Depending at which level you make the cut, you can decide how many clusters you want.

A great resource on this topic is Kaufman, L., & Roussew, P. J. (1990). "Finding Groups in Data - An Introduction to Cluster Analysis". John Wiley & Sons


What is the process of inventing deep neural network models? How researchers deal with long training times? After reading this topic on GitHub how long time it takes to train YOLOV3 on coco dataset I was wondering how researchers deal with long training times while inventing new architectures.
I imagine that to evaluate the model you need to train it first. How do they make tweaks in their architectures, e.g. tweaking layers, adding pooling, dropout, etc., if training can take a few days? Is it pure art and it is designed roughly or is it a more deliberate process?
What are the steps of engineering new architecture using deep neural networks?

Part of the answer is that new architectures are often variants of existing architectures. There are some rough heuristics people follow (e.g. using powers of 2 in layer sizes, changing layer sizes according to a schedule, adding normalization layers, etc.). However, even when building off an existing architecture, there can still be a lot of toying around with different configurations and hyperparameters to require copious training time.
Researchers working for a company will likely have the luxury of some serious compute, e.g. GPU clusters or cloud services like AWS. This can really make a huge difference over training a model on your PC GPU. Further, many models can be trained at once. Many academic institutions also employ cluster computing on dedicated GPU clusters - for example, here at the University of Utah our department has a cluster with 8 really excellent GPUs.
One last thing I will note, is that it's not always necessary to train the model to completion on the full training set to get a sense of which network configuration and hyperparameter settings are working better than others. If I have a massive dataset that will take days to train on, I may take a smaller subset of the training data and do some quick comparisons between different models to get a feel for what changes have the greatest impact on test performance. It's not completely reliable, but it can guide intuition to narrow down the number of models you want to train on the full dataset to convergence. For example, using this technique I was able to find that predictions from my model were fairly invariant to the exact layer sizes I was using and was far more sensitive to a hyperparameter in my optimization solver.


Would it be ethical to use AI to determine a user’s gender from the content they upload, without them knowing? I’ve been thinking about this for a few days and can’t tell if this would feel morally just to an average user

Ethics aside, I think this could potentially create a number of different issues. What if your algorithm guesses wrong and I'm stuck with a UI that's targeting incorrectly? What if my wife and I share an account? What if I'm a (insert orientation here) (insert gender here) who [coaches, supports] a [men's, women's] [volleyball, football] team? People are very diverse, and you could make many mistakes. Would you then provide tools for the user to correct these mistakes? How would you be able to do so without the user being offended?

Instead, I see many fewer issues that might arise from a section like the following:

  Let's get to know each other!

  If you'd like, we can help tune your profile to match your interests.

  You can start by selecting some of your interests from the list below, or try searching for your own.

  Fishing Cooking Soccer Video Gaming Social Media
  Search for my interest...

  Skip this section →

You could continue with other questions that might actually be relevant to your shaping your UI or application, allowing the user to omit/delete details for any or all questions.

Being purely an opt-in experience prevents any issues with the user feeling like your app may be "talking about them" behind their back.

In this case, specifically at this point in time, I think it's wise to be transparent about what data your application knows (or thinks it knows) about your users.


Is AI capable to replace pilots entirely on the commercial flights? Since we've self-driving cars already, would we have self-flying commercial flights in the near future? Basically the AI which can do take off, flying, landing and parking.

Self-flight is already running on all commercial airlines, but only at the cruising altitude. When you flight from London to New York, most of the time the pilots are monitoring the autopilot systems. Running AI on cruising altitude is arguably a simper problem than self-driving cars. The system has been in-place for years.

The real danger of airlines is taking off/landing. As you can imagine, the costs for a bad judgement could be quite serious. Anything could happen while the plane is still under the clouds. While our AI technology is improving, it will never replace human intervention, especially when the costs of collision is very high. If you don't believe me, google for how Facebook is hiring humans for reading "fake news". Facebook has publicity acknowledged AI will never be sufficient for them, then why would AI good enough for replacing well trained pilots?

AI on a commercial airplane is already taking place, but there is no possibility ever it will take over the world. We will always need pilots who knows how to work with the system, making important decisions.

Trained pilots + good AI assistance will make the world a better place.


How could you generate sentences from lists of facts Let's pretend we had a list of facts (similar to prolog tuples) that define some knowledge about some entities. e.g.

doing(clean, data)
done(collect, data)
todo(train, model)
todo(write, paper)

What methods could I use to generate sentences like:

You should be cleaning the data you collected, then you need to train your model and write your paper.

When you parse sentences you use grammatics to sentences to get the meaning (semantics).

Now you use sematics (tuples) and grammatics to make a sentence.

Your knowledge includes verb, noun and some kind of notion of readiness of the task defined by them.

To form sentences you also need knowledge / rules about the extra eligible words between these categories, as well as how time relates to verb grammatical tense.

You have to code a parser for tuples that codes the tuple in the three parts, formulate the grammatics,
and then apply the grammatics to the parsed tuple data.

After that you'll get the sentences.

More precise information on how to code that and this information plus the picture source:

http://cs.union.edu/~striegnk/courses/nlp-with-prolog/html/node93.html#l14.sec.nlg