Given enough graphical data, could you train an AI to plot a polynomial graph based on the input conditions? Good day everyone.

I am curious if it is possible for an AI to plot a time-series graph based on a single input.
Using free fall impact as an example.

Assuming we drop a ball from height 100m and record the force it receives relative to time.
We will get a graph that looks something like below.

Now we drop the ball from a height of 120m, record the forces, and we get another graph in addition to our original.

What I am wondering is:
If we have a large set of data on 60m to 140m (20m interval) height drops, would we be able to generate a regression model that plots the responses when given an arbitrary drop height? (i.e plot force response when dropped from 105m)

Thank you all very much for your time and attention.

Yes this is possible, using any machine learning approach that supports regression. You have two main approaches:

Input $h$ the height of the drop, multiple outputs, one per time offset that you want to plot. Each individual output calculates the predicted force at a specific offset time.
Inputs $h$ the height of the drop and $t$ a time offset, one output. The single output calculates the predicted force due to given height and at given time.

The main thing to bear in mind is that statistical learning techniques typically do not generate physics-like models. Test inputs close to training examples should generate reasonable graphs that interpolate between those from training data. Test inputs far away from the training examples (e.g. you train on data of drops from 60m to 140m, but use an input of 10m or 200m) will likely generate wildly incorrect outputs. The main exception to this is if your ML model includes some good guesses at the underlying physics model, in which case it is possible that a regression algorithm will tune the parameters of that model plus filter out terms that should not be part of the model, resulting in a system that extrapolates much better. That is very unlikely happen by chance, it requires up-front design.


Neural network to detect "spam"? I've inherited a neural network project at the company I work for. The person who developed gave me some very basic training to get up and running. I've maintained it for a while. The current neural network is able to classify messages for telcos: it can send them to support people in different areas, like "activation", "no signal", "internet", etc. The network has been working flawlessly. The structure of this neural network is as follows:

 model = Sequential()
 model.add(Dense(500, input_shape=(len(train_x[0]),)))
 model.add(Activation('relu'))
 model.add(Dropout(0.6))
 model.add(Dense(250, input_shape=(500,)))
 model.add(Activation('relu'))
 model.add(Dropout(0.5))
 model.add(Dense(len(train_y[0])))
 model.add(Activation('softmax'))
 model.compile(loss='categorical_crossentropy',
                  optimizer='Adamax',
                  metrics=['accuracy'])

This uses a Word2Vec embedding, and has been trained with a "clean" file: all special characters and numbers are removed from both the training file and the input data.

Now I've been assigned to make a neural network to detect if a message will be catalog as "moderated" (meaning it's an insult, spam, or just people commenting on a facebook post), or "operative", meaning the message is actually a question for the company.

What I did was start from the current model and reduce the number of categories to two. It didn't go very well: the word embedding was in spanish from Argentina, and the training data was spanish from Peru. I made a new embedding and accuracy increased by a fair margin (we are looking for insults and other curse words. In spanish a curse word from a country can be a normal word for another: in Spain "coger" means "to take", and in Argentina it means "to f__k". "concha" means shell in most countries, but in Argentina it means "c__t". You get the idea).

I trained the network with 300.000 messages. Roughly 40% of these were classified as "moderated". I tried all sorts of combinations of cycles and epochs. The accuracy slowly increased to nearly 0.9, and loss stays around 0.5000.

But when testing the neural network, "operative" messages generally seem to be correctly classified, with accuracy around 0.9, but "moderated" messages aren't. They are classified around 0.6 or less. At some point I tried multiple insults in a message (even pasting sample data as input data), but it didn't seem to improve.

Word2Vec works fantastically. The words are correctly "lumped" together (learned a few insults in Peruvian spanish thanks to it).

I put the neural network in production for a week, to gather statistics. Basically 90% of the messages went unclassified, and 5% were correctly classified and 5% wrong. Since the network has two categories, this seems to mean the neural network is just giving random guesses.

So, the questions are:

Is it possible to accomplish this task with a neural network?
Is the structure of this neural network correct for this task?
Are 300k messages enough to train the neural network?
Do I need to clean up the data from uppercase, special characters, numbers etc?

The Project Summarized

The project goal appears to be a common one: Routing correspondence in an efficient manner to maintain good but low cost customer and public relations. A few features of the project were mentioned.

Neural network project
Received some design and project history from predecessor
Classifies messages for telcos
Sends results to support groups at appropriate locales
Uses 2 relu layers, ending with a softmax
Word2Vec embedding
Trained with a clean language file
All special characters and numbers removed

The requirements for current development were indicated. The current work is to develop an artificial network that places incoming messages into one of two categories accurately and reliably.

Moderated — insulting, fraudulent in purpose (spam), trivial routine
Operative — relevant question requiring internal human attention

Research and development is beginning along reasonable lines.

Trained with 300,000 messages
Word2Vec used
40% of classified as moderated
Permuted cycles and epochs
Achieved 90% accuracy
Loss stays near 0.5
In test, operative accuracy 0.9, moderated accuracy max of 0.6

First Obstacle and Feasibility

The first obstacle encountered is that in QA using production environment data, 90% of the messages where left unclassified, 5% of the classifications were accurate, and the remaining 5% were inaccurately classified.

It is correct that the even split of 5% accuracy and 5% inaccuracy indicates that information learned is not yet transferable to the quality assurance test phase using real production environment messages. In information theory phraseology, no bits of usable information were transferred and entropy remained unchanged on this first experiment.

These kinds of disappointments are not uncommon when first approaching the use of AI in an existing business environment, so this initial outcome should not be taken as a sign that the idea won't work. The approach will likely work, especially with foul language, which is not dependent on cultural references, analogies, or other semantic complexity.

Recognizing notices that are for audit purposes only, from a social network accounts or purchase confirmations, can be handled through rules. The rule creation and maintenance can theoretically be automated too, and some proprietary systems exist that do exactly that. Such automation can be learned using the appropriate training data, but real time feedback is usually employed, and those systems are usually model based. That is an option for further down the R&D road.

The scope of the project is probably too small, but that's not a big surprise either. Most projects suffer from early overoptimism. A pertinent quote from Redford's The Melagro Beanfield War illuminates the practical purpose of optimism.

          APPARITION

  I don't know if your friend knows what he's in for.

          AMARANTE

  Nobody would do anything if they knew what they were in for.

Initial Comments

It is not necessary to reduce the number of message categories to two, but there is nothing wrong with starting R&D by refining approach and high level design with the simplest case.

The last layer may be more training efficient if a binary threshold is used for the activation function instead of softmax, since there is only one bit of output needed when there are only two categories. This also forces the network training objective to be the definitive selection of a category, which may benefit the overall rate of R&D progress.

There may be ways of improving outcomes by adding more metrics in the code to beyond just 'accuracy'. Others who work with such details every day may have more domain specific knowledge in this regard.

Culture and Pattern Detection

Insults and curse words are entirely different kinds of things. Foul language is a linguistic symbol or phrase that fits into a broadcasting or publishing category of prohibition. The rules of prohibition are well established in most languages and could be held in a configuration file along with the permutations of each symbol or phrase. In the case of sh*t, related forms include sh*tty, sh*thead, and so on.

It is also useful to distinguish the sub-sets of foul language.

Cursing (expressing the wish for calamity to befall the recipient)
Swearing (considered blasphemy by some)
Exclamations that are considered foul by publishers and broadcasters
Additional items parents don't want their children to hear
Edge cases like crap

The term foul language is a super-set of these.

Distribution Alignment

Learning algorithms and theory are based on probabilistic alignment of feature distributions between training and use. The distribution of training data must closely resembles the distribution found when the trained AI component is later used. If not, the convergence of learning processes on some optimal behavior defined by gain or loss functions may succeed but the execution of that behavior in the business or industry may fail.

Internationalization

Multilingual AI should usually be fully internationalized. Training and use of training with two distinct dialects will almost always perform poorly. That creates a data acquisition challenge.

As stated above, classification and learning depend on the alignment of statistical distributions between data used in training and data processing relying on the use of what was learned. This is also true of human learning, so this requirement will not likely be overcome any time soon.

All these forms of foul language must be programmed flexibly across these cultural dimensions.

Character set
Collation order
Language
Dialect
Other locale related determinants
Education level
Economic strata

Once one of these is included in the model (which will be imperative) then there is no reason why the others cannot be included at little cost, so it is wise to begin with standard dimensions of flexibility. The alternative will likely lead to costly branching complexity to represent specific rules, which could have been made more maintainable by generalizing for international use up front.

Insult Recognition

Insults require comprehension beyond the current state of technology. Cognitive science may change that in the future, but projections are mere conjecture.

Use of a regular expression engine with a fuzzy logic comparator is achievable and may appease the stakeholders of the project, but identifying insults may be infeasible at this time, and the expectations should be set with stakeholders to avoid later surprises. Consider these examples.

The nose on your face looks like a camel.
Kiss the darkest part of my little white. (From Avatar screenplay)

The word combinations in these are not likely to be in some data set you can use for training, so Word2Vec will not help in these types of cases. Additional layers may assist with proper handling of the at least some of the semantic and referential complexity of insults, but only some.

Explicit Answers to Explicit Questions

  Is it possible to accomplish this task with a neural network?

Yes, in combination with excellence in higher level system design and best practices for internationalization.

  Is the structure of this neural network correct for this task?

The initial experiments look like a reasonable beginning toward what would later be correct enough. Do not be discouraged, but don't expect the first pass at something like this to look much like what passes user acceptance testing a year from now. Experts can't pull that rate of R&D progress off, unless they hack and cobble something together from previous work.

  Are 300k messages enough to train the neural network?

Probably not. In fact, 300m messages will not catch all combinations of cultural references, analogies, colloquialisms, variations in dialect, plays on words, and games that spammers play to avoid detection.

What would really help is a feedback mechanism so that production outcomes are driving the training rather than a necessarily limited data set. Canned data sets are usually restricted in the accuracy of their probabilistic representation of social phenomena. None will likely infer dialect and other locale features to better detect insults. A Parisian insult may have nothing in common with a Creole insult.

The feedback mechanism must be based on impressions in some way to become and remain accurate. The impressions must be labelled with all the locale data that is reasonably easy to collect and possibly correlated to the impression.

This implies the use of rules acquisition, fuzzy logic control, reinforcement learning, or the application of naive Bayesian approaches somewhere appropriate within the system architecture.

  Do I need to clean up the data from uppercase, special characters, numbers etc?

Numbers can be relevant. Because of historical events and religious texts, 13 and 666 might be indications of something offensive, respectively. One can also use numbers and punctuation to convey word content. Here are some examples of spam detection resistant click bait.

I've got a 6ex opportunity 4u.
Wanna 69?
Values are rising 50%! We have 9 investment choices 4 you to check out.

The meaning of the term special character is vague and ambiguous. Any character in UTF-8 is legitimate for almost all Internet communications today. HTML5 provides additional entities beginning with an ampersand and ending with a semicolon. (See https://dev.w3.org/html5/html-author/charref.)

Filtering these out is a mistake. Spammers leverage these standards to penetrate spam detection. In this example, the stroke similarities of a capital ell (L) and those of the British pound symbol can be exploited to produce spam detection resistant click bait.

Do you like hot £egs?

Removing special characters that fit within the Internet standards of UTF-8 and HTML entities will likely lead to disaster. It is recommended not to follow that part of the predecessor's design.

Regarding emoticons and other ideograms, these are linguistic elements that may represent in text encoding the volume, pitch, or tone modulation of phonetics, or they may represent face or body language. In many languages ideograms are used in place of words. For a global system running in parallel with the blogsphere, emoticons are part of linguistic expression.

For that reason, they are not significantly different than word roots, prefixes, suffixes, conjugations, or word pairs as linguistic elements which can also express emotion as well as logical reasoning. For the learning algorithm to learn categorization behavior in the presence of ideograms, the ideograms must remain in training features and later in real time processing of those features using the results of training.

Additional Information

Some additional information is covered in this existing post: Spam Detection using Recurrent Neural Networks.

Since spam detection is closely related to fraud detection, the spammer fraudulently acting like a relationship already exists with their recipients, this existing post may be of assistance too: Can we implement GAN (Generative adversarial neural networks) for classication problem like Fraud detecion?

Another resource that may help is this: https://www.tensorflow.org/tutorials/representation/word2vec


How should I define an MDP for this problem where we need to guess a number and minimise the number of guesses? A number has randomly been chosen from 1 to 3. On each step, we can make a guess and we will be told if our guess is equal, bigger or smaller than the chosen number. We're trying to find the number with the least number of guesses.

I need to draw the MDP model for this question with 7 states, but I don't know how the states are supposed to be defined. Can anyone help?

When formulating a problem into an MDP you need to define the states, of the system, the possible actions you can take, the transition probabilities between states depending on the action and the rewards earned (or costs paid) for the state transitions. The important part here is to create a state-space that has a Markov property which in plain words means that in any given state you have enough information to make the optimal decision.

Rewards: in this case, I think it is fairly obvious that a positive reward should be given if we find out the number and zero reward if we don't.
Actions: the actions we can take is making a guess, that is the possible actions will come from the action-space A=(1,2,3).
States: this is a bit more difficult to come up with. Here the intuition should come from thinking about the actions you can make and how they change the information you have about the system. In our state-space, each state will represent the set of numbers that the answer might possibly be. For example in state the (1,3) we know that the guessed number is either 1 or 3. Also, we will denote these states with capital letters to simplify the state transitions.

A=(1, 2, 3): the initial state as at the beginning of the search the guessed number can be either of 1, 2 or 3;
B=(1, 2): the guessed number is either 1 or 2;
C=(1, 3): the guessed number is either 1 or 3;
D=(2, 3): the guessed number is either 2 or 3;
E=(1): the guessed number can only be 1;
F=(2): the guessed number can only be 2;
G=(3): the guessed number can only be 3;
H=(): final state, we found the guessed number.

State transitions: this is very simple, just imagine which states are reachable in a state depending on the guess (action) we make. Here the notation P(A|B,1) will mean the probability that we reach state A from state B given that we guessed 1. (Note: we will assume uniform distribution for the guessed number). I think I'll be lazy here and not write down the all the transitions as they get very repetitive once you understand how they are made, I'll just provide examples for all cases.

P(D|A,1)=2/3: we guess 1 in the initial state and we miss with 2/3 chance, the reply will be that the guessed number is greater than 1.
P(H|A,1)=1/3: we guessed right, so we reach the final state.
P(B|A,3)=2/3: similarly we guess 3 and miss.
P(H|A,3)=1/3: we guess 3 and we are right.
P(C|A,2)=0: if our guess 2 is wrong we get a reply of whether the guessed number is greater or smaller than 2 therefore we won't get into this state.
P(E|A,2)=1/3: we guess 2 and the reply is that the number is smaller than 2.
P(G|A,2)=1/3
P(H|A,2)=1/3
P(B|B,3)=1: guessing 3 in state B doesn't provide more information so we reach the same state with probability 1.
P(H|B,1)=1/2: 1 is the right guess with 1/2 probability in this state.
P(F|B,1)=1/2: similarly we miss with 1/2 chance if we guess 1 so we reach state F where we know what is the good solution however we haven't won yet as we still need to make one more guess to reach the final state.
P(F|F,1)=1: again guessing 1 in state F doesn't give us extra information so we get back to state F.
P(H|F,2)=1: however, guessing 2 in state F will give us the final state, as it is the right guess.

Note that I defined 8 states earlier, however, state C=(1,3) is never reached so we don't actually need it.

I hope this helped and you will be able to finish the rest.


How to start learning about artificial intelligence? I have been wanting to get started learning about artificial intelligence but I know almost nothing about coding or anything. So my question is, what would be the best way to get started in learning about artificial intelligence, as in should I learn some kind of coding language or is there some kind of other concept you need to know before getting started. So I'm just kind of looking for the best way to get started if you literally know nothing.

Read:

'Artificial Intelligence - A modern approach' by Russell and Norvig.

'Fluid Concepts and Creative Analogies' by Douglas Hofstadter.

'Machine Learning and Pattern Recognition' by Bishop

'The Emotion Machine' by Marvin Minsky


What are examples of machine learning techniques inspired by neuroscience? What are examples of machine learning techniques (i.e. models, algorithms, etc.) inspired (to different extents) by neuroscience?

Particularly, I'm interested in recent developments, say less than 10 years old, that have their basis in neuroscience to some degree.

There is a category of neural networks that more closely attempt to mimic biological neural networks by incorporating also time (i.e. not all neurons fire at the same time). They are called spiking neural networks (SNNs) and their name comes from the fact that they use spiking neurons (i.e. neurons that fire discrete signals and affect other neurons at possibly different times).

SNNs are mainly used in neuroscience, and aren't commonly used in machine learning because they currently have some apparent limitations (e.g. non-differentiability, so gradient descent and back-propagation can't be applied, but GD and BP aren't really biologically realistic anyway, although some people already tried to apply GD to SNNs) and their performance isn't still as good as the performance of traditional deep learning models, which make them not so appealing to the deep learning community (which is currently mainly driven by performance and utility). Nevertheless, the performance gap between traditional neural networks and spiking neural networks is decreasing. See Deep Learning in Spiking Neural Networks (2019) by Amirhossein Tavanaei et al. for more details.

There are already commercial implementations of a hardware-accelerated SNNs (e.g. BrainChip provides this service). These hardware-accelerators are often called neuromorphic chips (or processors) and all computing based on SNNs or processors that attempt to implement biological neural networks is known as neuromorphic computing.

There's also the related area called reservoir computing, which studies neural networks (such as liquid-state machines or echo state machines) that make use of reservoirs (which are fixed during learning) to attempt e.g. to improve training efficiency. See An overview of reservoir computing: Theory, applications and implementations (2007) by Benjamin Schrauwen et al. for an overview.

Numenta (and, in particular, Jeff Hawkins, the founder of Numenta and author of an interesting book called On Intelligence) has also been studying neuroscience for a long time in order to develop models and theories of human intelligence. They call their new theory The Thousand Brains Theory of Intelligence, which is inspired by biological grid cells. This is also related and similar to capsule networks (often associated with Hinton).


How many layers and neurons on the layers does a neural network need for for sales prediction? The inputs (features) and expected output for my ANN are these:

Input 1: Product id (number, cast to double)
Input 2: Year in the past (1900..2017, cast to double)
Input 3: Month of year (1..12, cast to double)
Expected output: Sale of month (number of units sold, cast to double)

I need to predict the sale of a product for a certain month in a certain year. How many layers and how many neurons on there layers should I put?

Every answer you get is just an opinion, it is based on experience.
My answer is a single hidden layer of 5 neurons.

Besides, I recommend you to use a TWEANN system (Topology and Weight Evolving ANN).

Such system applys genetic algorithm to search and optimize an ANN with the optimal topology and weights.

Take a look at NEAT and DXNN.


What are support values in a support vector machine? I started reading up on SVM and very little is defined of what are support values. I reckon it's they are denoted as $\alpha$ in most formulations.

In the least-squares SVM (LS-SVM) the non-zero Lagrange multipliers ($\alpha$) are the support values.  The corresponding data points are the support vectors.  Johan Suykens explains this in Least Squares Support Vector Machines.


How can it be shown clearly and transparently that the outcomes of data-driven health and care technology are validated? The National Health Service (NHS) wrote down several principles in a document Code of conduct for data-driven health and care technology (updated 18 July 2019). I am concerned with principle 7.

Show what type of algorithm is being developed or deployed, the ethical examination of how the data is used, how its performance will be validated and how it will be integrated into health and care provision
Demonstrate the learning methodology of the algorithm being built.
Aim to show in a clear and transparent way how outcomes are validated.

But how exactly can outcomes be shown in a clear and transparent way how outcomes are validated?

The current version of the document Code of conduct for data-driven health and care technology provides more details about all principles, including principle number 7, which I will quote here.

Consider how the introduction of AI will change relationships in
health and care provision, and the implications of these changes for
responsibility and liability. Use current best practice on how to
explain algorithms to those taking actions based on their outputs.
When building an algorithm, be it a stand-alone product or integrated
within a system, show it clearly and be transparent of the learning
methodology (if any) that the algorithm is using. Undertake ethical
examination of data use specific to this use-case. Achieving
transparency of algorithms that have a higher potential for harm or
unintended decision-making, can ensure the rights of the data subject
as set out in the Data Protection Act 2018 are met, to build trust in
users and enable better adoption and uptake.
Work collaboratively with partners, specify the context for the
algorithm, specify potential alternative contexts and be transparent
on whether the model is based on active, supervised or unsupervised
learning. Show in a clear and transparent specification:

the functionality of the algorithm
the strengths and limitations of the algorithm (as far as they are known)
its learning methodology
whether it is ready for deployment or still in training
how the decision has been made on the acceptable use of the algorithm in the context it is being used (for example, is there a
committee, evidence or equivalent that has contributed to this
decision?)
the potential resource implications

This specification and transparency in development will build trust in
incorporating machine-led decision-making into clinical care.


Can we calculate mean recall and precision I'm evaluating the accuracy in detecting objects for my image data set using three deep learning algorithms. I have selected a sample of 30 images. To measure the accuracy, I manually count the number of objects in each image and then calculate recall and precision values for three algorithms. Following is a sample:

Finally to select the best model for my data set, can I calculate the mean Recall and mean Accuracy? For Example:

For the precision metric for example you have:

$$
 Precision = \frac{TP}{TP+FP},
$$
with TP = True Positive and FP = False Positive.

Imagine you have the following values:
Image 1: $TP = 2, FP = 3$
Image 2: $TP = 1, FP = 4$
Image 3: $TP = 3, FP = 0$

The precision scores as you calculated will be:
Image 1: $2/5$
Image 2: $1/5$
Image 3: $1$
Your average will be: $0.533$

On the other hand if you sum them all up and then calculate the precision value you get:

$P = \frac{6}{6+7} = 0.462$

This proves that averaging the precision scores is not the same as calculating the total precision in one go.

Since what you want is to know how precise your algorithm is, independently of the precision for each image, you should sum all the TP and FP and only then calculate the precision for each model. This way you will not have a biased average. The average would give the same weight to an image with a larger number of objects as to another image which had fewer objects.


Who are thought leaders in the chatbot space? I'm in the process of learning as much about chatbots/CUI applications as possible and I'm trying to find more information on some of the major players in this field. By this, I mean any execs, developers, academics, designers, etc. who are doing cutting edge things. Some examples could be David Marcus (VP of messaging products at Facebook) or Adam Cheyer (VP of engineering at Viv).

One of them is certainly Doctor Richard Wallace.  Doctor Wallace was the original author of the Artificial Intelligence Markup Language spec and is Chief Science Officer at PandoraBots.


Why does PCA of the vertices of a hexagon result in principal components of equal length? I do PCA on the data points placed in the corners of a hexagon, and get the following principal components:

The PCA variance is $0.6$ and is the same for each component. Why is that? Shouldn't it be greater in the horizontal direction than in the vertical direction? The data is between $-1$ and $1$ in the $x$-direction but only between $-\sqrt{3}/2$ and $\sqrt{3}/2$ in the $y$-direction. Why PCA results in the equal length components?
The length of each vector in the picture is the twice the square root of the variance.
UPDATE: added more points, the variances changed to $0.477$ but still they are equal.

UPDATE 2: Added even more points, the variances changed to $0.44$ but still they are equal.

Assuming that the $6$ vertices of the hexagon are on the unit circle,
>>> from sympy import *
>>> A = Matrix([[ 1, Rational(1,2),-Rational(1,2), -1, -Rational(1,2), Rational(1,2)],
                [ 0,     sqrt(3)/2,     sqrt(3)/2,  0,     -sqrt(3)/2,    -sqrt(3)/2]])
>>> A * A.T
Matrix([[3, 0],
        [0, 3]])

Since ${\rm A} {\rm A}^\top - 3 \, {\rm I}_2 = {\rm O}_2$, any two orthogonal directions could be the principal components.


What are examples of approaches to dimensionality reduction of feature vectors? Given a pre-trained CNN model, I extract feature vector of images in reference and query dataset with several thousands of elements.

I would like to apply some augmentation techniques to reduce the feature vector dimension to speed up cosine similarity/euclidean distance matrix calculation.

I have already come up with the following two methods in my literature review:

Principal Component Analysis (PCA) + Whitening
Locality Search Hashing (LSH)

Are there more approaches to perform dimensionality reduction of feature vectors? If so, what are the pros/cons of each perhaps?

Dimensionality reduction could be achieved by using an Autoencoder Network, which learns a representation (or Encoding) for the input data. While training, the reduction side (Encoder) reduces the data to a lower-dimension and a reconstructing side (Decoder) tries to reconstruct the original input from the intermediate reduced encoding.

You could assign the encoder layer output ($L_i$) to a desired dimension (lower than that of the input). Once trained, $L_i$ could be used as a alternative representation of your input data in a lower feature-space, and can be used for further computations.


Is there any way can teach AI creative painting (not convert photo to paint)? I already know that AI can paint, by using genetic algorithms, there are already lots of works such as this and this. In addition, I also know AI can compose: Song from PI: A musically plausible network for pop music generation (genetic algorithm too).
But what I find interesting is not painting those ambiguity/abstract paintings.
The not abstract painting flow I think is (just for example):

at least training AI with superman's comic

give AI a very simple posture sketch of a standing human

AI paint it to superman.

Currently, I don't know if there is any way/guide/thought/algorithm can teach AI to paint a superman like the comic (not abstract ones). I'd like to research this area but can't find where and how to start.

This paper (that was featured in another question), StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks describes some techniques similar to what you described in the question.  Instead of comics, the system in the paper is trained on a database of nature photos (birds or flowers) combined with text descriptions of each photo.  It will then draw an "imaginary" bird or flower based purely on a text description.  It does this in two steps: it the first, it picks a posture for the depicted subject (I believe randomly...) and roughs out patches of color that match the described features of the subject in the selected posture.  It then refines the sketch, again using the text description for parameters, until it appears "photo-realistic" in the style of the training photos.  It does not just recreate its training images; instead, it is able to recreate features of its training images with different positions/colors.


A way to give more weight to particular data? Let us for these purposes say with are working with any feed forward neural network.

Let us also say, that we know beforehand that certain portion of our dataset arsignificantly more impactful or important to our underlying representation. Is there anyway to add that “weighting” to our data?

Yes, You just weight the entries of loss function like sum( Wi(li-t2)**2) for MSE. Most frameworks provides this ability. For example in keras fit method has a parameter - sample_weight: Optional Numpy array of weights for the training samples.


What is the neuron-level math behind backpropagation for a neural network? I am quite new in the AI field. I am trying to create a neural network, in a language (Dart) where I couldn't find examples or premade libraries or tutorials. I've tried looking online for a strictly "vanilla" python implementation (without third-party libraries), but I couldn't find any.

I've found a single layer implementation, but it's done only with matrices and it's quite cryptic for a beginner.

I've understood the idea between the feed forwarding, a neuron calculates the sum of its inputs, adds a bias and activates it.

But I couldn't find anything a neuron-level explanation of the math behind backpropagation. (By neuron-level I think of the math down to the single neuron as a sequence of operations instead of multiple neurons treated as matrices).

What is the math behind it? Are there any resources to learn it that are suitable as a beginner?

Backpropagation is actually a lot easier than it is made out to be - if you have a basic understanding of calculus and the chain rule, and the single multi-variable calculus rule that to combine 2 gradient vectors, you simply add them.

This is hands down the best walk through of back prop I've found on the internet. If you are still confused after that, feel free to ask me any further questions. Here is also a quick forward and backward pass example I made for a simple CNN (only a few layers though, and the gradient only goes back to channel 1 of filter 1)


Is there a name for this principle? I have an idea for a transformative decision rule (TDR), but I want to know if there's already a name for it.

  Any ordinal scale should be converted to a scale such that all  the outcomes are assigned consecutive values.

Let v be the function that assigns values to outcomes (defined as ordered pairs of acts and states).

Let x and y denote any particular such ordered pairs.
Let P be the preference.
Let X+ be the set of all outcomes that are more preferred than x (similarly also for y) X+ in P.

  If x in Y+, and x is the closest outcome to y in Y+, then v(x) = v(y) + 1 for all x,y in P.

I could have explained it more formally mathematically, but Latex doesn't seem to render here. If there's no name for it, I'm thinking of "The Law of Consecutive Ordinals (LCO)". As for why I support the rule, it's because it's a rule that needs to be applied to an ordinal scale before I apply an effective decision rule (EDR) that I thought of for ordinal scales. I want to know if there's a name for that principle.

I'd describe this as a Ranking.

However, it's not always possible to distil a set of outcome preferences to a transitive metric, this is one of the central axioms of game theory. The Transitivity Axiom so the problem is relatively well understood (that is, at least, if I've understood your problem)


What are examples of AI that use philosophy derived ontologies? I'm looking for AI systems or natural language processors that use, in the classification and interrelation of notions/objects, some philosophical system, like basic laws of logic, Kantian or empiricism.

I have also read about goal-seeking procedures. Are these based on psychology fields and some particular psychology theory or these are ad hoc experiments, with only general terms applied?

Excellent question!  I'm currently working on the pre-Socratics as the most basic philosophies for first principles (These philosophers are intriguing for their simplicity and universality, and the "dawn of consciousness", in some conceptions, may be ascribed to the Classical Era.  Linguistically, ancient Greek is fundamental to meaning in the West.)

However, I was pointed at this very interesting blog post "Algorithms of the Mind" which has some useful links, and discusses Kant, which you may find useful.


Why do value iteration and policy iteration obtain similar policies even though they have different value functions? I am trying to implement value and policy iteration algorithms. My value function from policy iteration looks vastly different from the values from value iteration, but the policy obtained from both is very similar. How is this possible? And what could be the possible reasons for this?

Both value iteration (VI) and policy iteration (PI) algorithms are guaranteed to converge to the optimal policy, so it is expected that you get similar policies from both algorithms (if they have converged).

However, they do this differently. VI can be seen as truncated version of PI.

Let me first illustrate the pseudocode of both algorithms (taken from Barto and Sutton's book), which I suggest you get familiar with (but you are probably already familiar with them if you implemented both algorithms).

As you can see, policy iteration updates the policy multiple times, because it alternates a step of policy evaluation and a step of policy improvement, where a better policy is derived from the current best estimate of the value function.

On the other hand, value iteration updates the policy only once (at the end).

In both cases, the policies are derived from the value functions in the same way. So, if you obtain similar policies, you may think that they are necessarily derived from similar final value functions. However, in general, this may not the case, and this is actually the motivation for the existence of value iteration, i.e. you may derive an optimal policy from an non-optimal value function.

Barto and Sutton's book provide an example. See figure 4.1 on page 77 (p. 99 of the pdf). For completeness, here's a screenshot of the figure.


Could an AI feel emotions? Assuming humans had finally developed the first humanoid AI based on the human brain, would It feel emotions? If not, would it still have ethics and/or morals?

There is much discussion in philosophy about inner language and the ability to perceive pain (see Pain in philosophy article). Your question is in the area of philosophy and not science. If you define emotion as some state then you can construct simple automata with two states (emotion vs no-emotion). It can be a very complicated state with degrees of truth (percentage of emotion).

Basically, to mimic human emotion you need to make a living human-like organism, and still with todays understanding and technology you will not be able to recognize emotion in it. The only thing you can do is trust when it says "I'm sad". Now we are in the area of the Turing test, which is again philosophy, and not science.


Why is creating an AI that can code a hard task? For people who have experience in the field, why is creating AI that has the ability to write programs (that are syntactically correct and useful) a hard task?

What are the barriers/problems we have to solve before we can solve this problem? If you are in the camp that this isn't that hard, why hasn't it become mainstream?

AI has been applied to programming (check out TabNine, my favorite autocomplete engine) although not in as robust a fashion as you describe.

Programming requires a high level of abstract while AI is typically trained to solve a very specific task. Given thousands of examples of insert sort in Python I think a model could be trained (perhaps after autocomplete and syntax correction) figure it out. However at this point the field has not developed a more general intelligence that can apply the ideas of the algorithm to other problems.

Addition based on comments:

Big picture, training an algorithm to solve a general class of problems (say, web dev) requires a huge number of examples or an immense number of trials. Further, as the complexity of the problem grows the number of parameters necessary to build the model grows. Writing code is a very complex problem and would thus require a huge amount of data and a huge number of parameters making it totally infeasible with today's math and (because of how the math is solved) hardware.

Modern AI is has a very simple goal: find the model that solves a problem optimally. If we could quickly search every possible model this would be simple. Fields like machine (deep) learning and reinforcement learning are concerned with finding a good solution in a reasonable amount of time. At this point no such solution exists for a problem of such complexity.


Para Generation and Drawing Conclusion from X give Articles I was wondering any examples of the following;

para generation: For eg, given X similar paragraphs,  are you able to build a model to learn the style and generate a new para that is a paraphrase of the X paras. Similar in meaning but diff wording.
drawing conclusions from X given articles. He has a list of conclusions, check the X articles can provide evidence to the conclusions. Eg, given conclusion “city is not safe”, look for evidence such as “murders” and “thefts”.

Glady Appreciate,
Betty

Paraphrase generation would be made by an abstractive text summarization tool like Tensorflow as described in Google's blog.

Abstractive tool summarizes the text but also it adds extra words to make text more human like.

About the second problem scenario I would say the answer depends quite much what you mean by the conclusions and how you think the conclusions would be made. I strongly believe this falls in the category of artificial general intelligence, which is emerging area of AI, supported by association AGI, where the focus is on strong artificial intelligence and more human like features of AI.

Judging about conclusions at least needs a vast knowledge base of the semantics of different words and pieces of text in general combined to possible conclusions and their legitimate implications.


Agents meeting in a directed connected graph I've just started learning AI and I have a certain problem:

We have directed connected graph, where nodes are places one can go to and edges are the "roads" between places. We have K agents whose goal is to meet in one node. Agents start in different nodes. Note that more than one agent can be at one node at the same time and that all agents move by one node at every turn(they move synchronously).

We have to variants of this task:

1) in each turn every agent must move

2) an agent may pass on moving.

For a chosen variant I have to find an algorithm to complete this task, but it cannot be the searching state-space algorithm.

I've been sitting on this for a while but I cannot think of anything.

I've been thinking if agents could know each other positions in order to choose where to go, but it's searching state space. I also thought that if agents meet, they could continue together. But I'm looking for an alternative to searching state-space algorithms.

It's not possible to solve version 1) of the problem in general. To see why, consider a graph with 2 cities, and 2 agents, where the agents start in opposite nodes. Since both agents need to move every turn, they will never meet in the same city.

For version 2), I'm going to make some assumptions that aren't completely clear from your text:

Agents have only local information. They can't see the global structure of the graph, and just all agree to meet at city X, because they don't know that X exists. They also can't see where other agents are initially.
Cities are labelled, or otherwise identifiable to the agents. That is, an agent can tell whether they've visited a city before or not.
Agents can tell how many agents are in their current city, and in all adjacent cities.
There are a finite number of cities, and it's always possible to get from city A to city B, for any pair of cities (A,B).
Agents have no way to communicate with each other, except by being present in the same space or not.

In this setup, the following heuristic will always work:

If you've never seen another agent, move to a random adjacent city, or stay in place, with a uniform probability of taking each possible action.
If you've never seen another agent before, but you see one or more agents now, then switch permanently to the following strategy:

If you can see an adjacent city with more agents than your city, move there now. If you can see several such cities, move to the largest one. Pick randomly in case of a tie.
If an adjacent city has the same number of agents as yours, flip a coin to decide whether to go or stay. Eventually, one city ends up with more than the other, and then step 1 causes all agents to end up in the same city.
Otherwise, if you can see other adjacent cities with fewer agents, just stay put. They'll come to you.
If all adjacent agents are empty and there's a total of $n$ agents you can see in your city right now, then remember $n$.
As long as you still can't see more than $n$ agents in total, stay put with probability $(n-1)/n$ and move to a random adjacent city with probability $1/n$.
If you stayed put per step 5, and you still can only see n agents, and all but one of them is in your current city, move to the city with the single agent who moved. Otherwise, if 0 or more than 1 agent moved, stay put. The agents who moved come back, and we go to step 5.
After successfully moving, go to step 1 again.

You can see that once agents meet up in a group, that group never shrinks. Further, groups of agents are always trying to move around together. You can prove that a group moves an average of about $\frac{1}{2e}$ every step. It may take a while, but eventually every group will bump into every other group, resulting in all the agents being in the same city.


How to use the LSTM layer in PPO architecture? What is the best way of using the LSTM layer in PPO architecture?
Should I use them in the first layer of both actor and critic, or use them just before the final layer of these networks?
Should I feed the architecture with a stack of states (the state stacked with the k previous states)?

Not sure if you're using TensorFlow-Agents for this, but if you are, there's some nice functionality for this built out using the ActorDistributionRnnNetwork and ValueDistributionRnnNetwork that I've found to be very helpful.  The LSTM architecture is used in both the actor and critic networks.  See this page for more information - hope this helps!


Code freezes and never returns when linear_kernel (sklearn.metrics.pairwise) is used on 20M Movielens dataset I'm fairly new to ML/AI, i'm trying learn the content based recommendation - here is my source code - https://github.com/jaganlal/content-based-recommender

I'm using MovieLens 20M dataset - tags.csv to recommend similar movies based on its tag. But whenever i run (from sklearn.metrics.pairwise)

cosine_similarities = linear_kernel(tfidf_matrix, tfidf_matrix)

The python cell keeps executing and doesn't return at all. The code can be found at https://github.com/jaganlal/content-based-recommender

Am i doing anything wrong, any help is highly appreciated.

Thanks,

Jagan

A kernel is a precomputed distance function. This means that instead of computing the distances between pairs of points every time you need them, you'll just compute them all a single time, at the start, and cache the values in memory. This is a great idea if the dimensionality of your data is very high, and you expect to have to make a lot of distance computations. For example, if each vector was an high definition image with millions of pixels, computing the distance between two vectors would cost millions of operations.

A problem with this approach is that if you have many exemplars in your dataset, computing and storing the distances between every pair will become hugely expensive. In your case, you have 20 million exemplars. This means you have ~$20,000,000^2$ = $4*10^14$ pairs of distances to compute. If your computer does ~$10^9$ computations per second (as is normal these days), this will take you about $10^5$ seconds, or a bit more than a whole day. Much worse than this, you will need approximately 400TB of RAM to store the kernel. If you don't have this, your system will eventually start writing to disk, which will cause the calculation to take about 3 years instead of 1 day.

The solution here is not to do quadratic work with millions of datapoints. Either use a subset of the data, or try approximate distance methods.


Reinforcement Learning algorithm with rewards dependent both on previous action and current action Problem description:
Suppose we have an environment, where a reward at time step $t$ is dependent not only on the current action, but also on previous action in the following way:

if current action == previous action, you get reward = $R(a,s)$
if current action != previous action, you get reward = $R(a,s) - \text{penalty}$

In this environment, switching actions bears a significant cost. We would like the RL algorithm to learn optimal actions under the constraint that switching action is costly, i.e. we would like to stay in selected action as long as possible.
The penalty is significantly higher than an immediate reward, so if we do not take it into account, the model evaluation will have a negative total reward with almost 100% probability, since the agent will be constantly switching and extracting rewards from environment that are smaller than the cost of switching actions.
Action space is small (2 actions: left, right). I'm trying to beat this game with PPO (Proximal Policy Optimization)
Questions

How one might address this constraint: i.e. explicitly make the agent learn that switching is costly and it's worth sitting in one action even if immediate rewards are negative?

How can you make the RL algorithm learn that it's not the reward term $R(a_t|s_t)$ that is negative, and thus decreasing $Q(a_t|s_t)$ and $V(s_t)$, but it's the penalty term (taking the action that is different from the previous action at step $t-1$) that is pushing total reward down?

The answer to both your concerns is:

Add the previous action choice to the state representation.

It is all you need to do. It gives the agent the data it needs to learn the association of negative reward from not matching the previous action.
By making this data part of the state, you re-establish the Markov property in the MDP model of the environment, which you had otherwise lost by making the reward dependent on a variable that was both systematically changing and hidden from the agent.
The state in a MDP is often not just the current observations that the environment provides, but can include any relevant knowledge that the agent has. At the extreme that can include a complete history of all observations and actions taken to date. It is common practice to derive the state as a summary of recent history of observations and actions taken so far. In your case, all you need do is concatenate the previous action to the observation, because you know about the constraint and how it affects optimisation.


Neural Network example I read a lot about this, I understand how it work, but I would like the most simple example you can provide me, because I have no clue how I would make it in code. No matter the language( I would appreciate if it's derived from c), because I'm not here to copy-paste, and understand the essence.

I think this website is the best c-based neural net code for beginners.


Approach to classify a photo and extract text from it I am trying to make a personal ML project where my objective is using a photo from an invoice, for instance, a Walmart invoice, classify it as being a Walmart invoice and extract the total amount spent. I would then save this information in a relational database and infer some statistics about my spendings. The goal would be to classify invoices not only from Walmart but from the most frequent shops where I spend money and then extract the total amount spent. I already do this process manually, I insert my spendings in a relational database. I have a bunch of photos from different invoices that I have recorded over the past year for this purpose (training a model).

What algorithms would you guys recommend? From my point of view, I think that I need some natural language processing to extract the total amount spent and maybe a convolutional neural network to classify the invoice as being from a specific store?

Thanks!

There are three functional aspects to this project, each of which could be a sub-project, even though they are interrelated. The decoupling of these aspects in the associated R&D will likely improve the rate of development.

Classify images — in this case, the vendor brand
Locate and orient the text
OCR the text — in this case, the total

These are certainly feasible in the particular case of invoices from vendors, so the entire project certainly is.

The goal is to produce two fields of information from input images.

Vendor code
Total in some specific monetary unit

That past invoices, the names of the vendors, and the totals of each invoice are available as training data may be helpful. Whether the volume of training data is sufficient can only be determined through a very complicated application of theory to some of the data metrics or by trial. It is recommended to do both.

  What algorithms would you guys recommend?

It would be irresponsible to define an AI design in terms of what people recommend for algorithm. The algorithms that guys recommend are already covered by the path names for files in the algorithms directory paths, the example's directory paths, and the sections of the documentation of any good AI framework. Let's first talk about the design and then the algorithm options, so there is more of a basis for algorithm selection than the largely random reading selections of the members of a site.

The models involved and which can be parameterized such that they can be tuned through training are partly defined by the three sub-projects above.

Analysis

For this project, there is no reason for the AI to recognize which text on the invoice is the vendor name. If there are a hundred vendor invoice types, the speed in which a single person can identify the key boundaries of each invoice type is orders of magnitude faster than the work necessary to develop a completely general algorithmic approach for automating that work. It would only be resource thrifty to develop such automation if there were thousands of vendors and constantly varying templates for invoices, which is likely in the case of personal finance.

These are the key bounds of the form quadrilaterals (four sided polygons) within which the following objects lie.

Vendor name text box
Total amount text box
Rectangle around the entire form

Digitizing the twelve points through one of the labelling programs is a model much more likely to produce a reliable system than the representation of three rectangular bounding boxes and tilts. This is because tilt angle requires at least two adjacent points anyway and the aspect ratio cannot be held constant in real scanning scenarios when the scanner might be replaced or produce different ratios with wear or the smoothness or wear characteristics of the paper invoice.

Doing all three things using a network would require more than one person's invoices for a decade, unless the person is a billionaire compulsive buyer with a team of secretaries doing scanning and data entry.

The models are then these.

Digitization of a quadrilateral document with varying contrast, brightness, tilt, horizontal pixels per inch, vertical pixels per inch, relative location and size of vendor name, and relative location and size of invoice total
Numeric characters and other monetary characters in a rectangular box
Brand image for the vendor, which may include a logo and type of arbitrary and possibly unique font

Now we can talk about artificial network approaches.

Adjusting for contrast, brightness, tilt, resolution, locations, and sizes will require a customized input, loss function, and layer arrangement based on the geometry involved. A GRU network may gain some advantages if the documents can be ordered chronologically, since the collection then becomes a time series, the trends of which can be exploited.
Monetary values can best be done using OCR libraries.
Recognizing the brand is probably best done with CNN as a categorization machine.

The system must indicate if the vendor and total are not found, in which case a new template is indicated, and the twelve points must be digitized for this new type.

It is possible to do this with one single deep convolutional network, but, again, the data set would need to be augmented. The one other way to do this is to create a filled-out-invoice generator to produce the volume of stochastically variable data across the various variability dimensions listed above to train the deep CNN.


How does AlphaZero's MCTS work when starting from the root node? From the AlphaGo Zero paper, during MCTS, statistics for each new node are initialized as such:

${N(s_L, a) = 0, W (s_L, a) = 0, Q(s_L, a) = 0, P (s_L, a) = p_a}$.

The PUCT algorithm for selecting the best child node is $a_t = argmax(Q(s,a) + U(s,a))$, where $U(s,a) = c_{puct} P(s,a) \frac{\sqrt{\sum_b N(s,b)}}{1 + N(s, a)}$.
If we start from scratch with a tree that only contains the root node and no children have been visited yet, then this should evaluate to 0 for all actions $a$ that we can take from the root node. Do we then simply uniformly sample an action to take?
Also, during the expand() step when we add an unvisited node $s_L$ to the tree, this node's children will also have not been visited, and we run into the same problem where PUCT will return 0 for all actions. Do we do the same uniform sampling here as well?

I looked at the Python pseudo-code attached to the Data S1 of the Supplementary Materials of the AlphaZero paper. Here is my findings:

Contrary to the paper, AlphaZero does not store $\{N(s, a), W(S, a), Q(s, a), P(s, a)\}$ statistics for each edge $(s,a)$. Instead, AlphaZero stores $\{N(s), W(S), Q(s), P(s)\}$ statistics for each node $s$.
When a leaf node $S_L$ is expanded, it's visit count, value scores, and action policies are immediately updated in $\{N(s), W(S), Q(s), P(s)\}$, so $N(s)$ is at least $1$. This is why in the paper, the backprop step updates for all time steps $t \le L$ rather than $t < L$. It makes sense to update $s_L$ even though there is no corresponding $a_L$ to pair it with.
Therefore, when a new leaf node is expanded, the value $U(s, a)$ of a child of that leaf node will be nonzero, since $\sqrt{\sum_b N(s,b)}$ is actually computed as $N(s_{parent})$ in the code, which is at least 1.
Oddly enough, I think there might be a bug in the pseudocode, because at the beginning on the first iteration (starting at the root node), $U(s,a) = 0$ for all child nodes of the root node. This is because at the first iteration, $N(s_{root}) = 0$. The value of all child nodes will be $0$, and since the authors chose to break ties according to Python's max function, the algorithm simply chooses the first element it finds in case of a tie.
After the first iteration, $N(s_{root}) > 0$ and so $U(s,a) \neq 0$ and things proceed as normal since the backprop step will have updated the visit count of the root node. So this possible bug/unintuitive behavior only affects the first iteration. It is extremely minor and insignificant, and does not affect the outcome of the MCTS, which is probably why it went unnoticed.


DQN Breakout adding an extra negative reward to help training? I'm trying to train a DQN, so I'm using OpenAI gym and Breakout (Breakout-v0).

I have altered the reward supplied by the environment: If the episode is not completed fully, the agent gets a -10 reward. Could be this counterproductive for learning?

In general, the approach of adding or altering the reward structure of a RL environment, when you are still trying to create an agent that solves the original problem, is called Reward Shaping.

Reward Shaping can be tricky to get right. In your case I think it may be counter-productive. The main issue is that the agent does not know the current time step as part of the state. So from its perspective, a negative reward for "timing out" actually looks like a random chance of getting that same reward in the kind of states that happen later in the game. The effect may be small enough chance that it doesn't really matter (after all it still gets positive rewards for hitting bricks). However, it may appear as high variance on certain game positions, making the agent learn to avoid some of them for no good reason.

If the aim here is to try and get the agent to speed up and finish an episode, a simpler trick might be to reduce the discount factor (e.g. from $0.999$ to $0.99$). This will cause the agent to focus on getting more short-term rewards, at the expense of long-term planning. In some environments this could be a problem of a different kind, but when the rewards are not sparse and there are not any special high-reward states that need extended setup, it should be OK.

If the aim is to punish losing a "game life" more severely, then this is less likely to cause a problem (because the state will clearly show what to expect), although it may change what the optimal behaviour is, or how it is approached. In general the impact would be reduced risk-taking, and even though Breakout is a deterministic game, the agent is presented with a stochastic environment because each action lasts a random number of frames (2-4). With a strong penalty for losing a life, I think that the agent will be less likely to try to get high bounce angles by hitting with extreme edge of the bat. Note that you don't need to punish the agent for losing a life, it learns that because the episode ends and it cannot score any more points. Value functions are all about predicting future rewards, so the value of any life-losing state is always $0$ anyway.

If you have more time, then there are also several extensions to basic DQN that require less sampled experience to learn an environment, or improve the resulting policies. A good source for those ideas is the paper Rainbow: Combining Improvements in Deep Reinforcement Learning.


Can the Raspberry Pi do audio learning? I want to create a project where my Raspberry Pi Micro-controller can listen to my parrot talking and then talk back to him. I would need my rpi to listen to some of his phrases at least 100 times so that the pi could recognize that certain phrase and then say it back to him and also to know when to use it to talk back to him. I would also like to use it to teach him new phrases and new sounds. I would like to know if this is even possible and what hardware and software I would use to go about this?

I have already asked this in the Raspberry Pi stack exchange but have recieved no answers.

Typically what you probably would want to do is do the training on something other than a Raspberry Pi. I think for what you're wanting to accomplish with having a computer talk back to your parrot, you won't need anything too crazy with a bunch of GPU's - but I don't think you'll want to necessarily do the training on a Pi either.

Here are some questions I have:

What are you going to plat back to the parrot? i.e. are you going to play it back random parrot sounds you found online or sounds that you've recorded? are you going to play it back what it just said? are you going to play it back a modification of what it just said?
Do you want it to respond to the parrot anytime the parrot speaks? Or when the parrot "says" something specific?

I think depending on the answers to those two questions, there are a couple of different paths that you could go down.

As for the hardware of the Raspberry Pi itself, I have never done any speech recognition with it, but I have done image recognition with it via the Movidius Neural Compute Stick which according to this Quora post, may be able to be used to offload some of the processing "relatively easily".

Here are some other links you may find valuable:

Tensorflow speech recognition: https://github.com/pannous/tensorflow-speech-recognition

The YouTube video that the above repo references: https://www.youtube.com/watch?v=u9FPqkuoEJ8

How to control a raspberry pi with your voice: http://www.techradar.com/how-to/how-to-control-the-raspberry-pi-with-your-voice
Voice recognition software setup on Raspberry Pi: https://diyhacking.com/best-voice-recognition-software-for-raspberry-pi/


What is the most abstract concept learned by a deep neural network? It seems that deep neural networks are making improvements largely because as we add nodes and connections, they are able to put together more and more abstract concepts. We know that, starting from pixels, they start to recognize high level objects like cat faces, chairs, and written words. Has a network ever been shown to have learned a more abstract concept that a physical object? What is the "highest level of abstraction" that we've observed?

You can train DNN to learn compute any abstract concept just by making that abstract concept as the label (output) in the training dataset. For example there are projects which detects emotions from peoples photos.


How can I model and solve the Knight Tour problem with reinforcement learning? I've read about the Knight Tour problem. And I wanted to try to solve it with a reinforcement learning algorithm with OpenAI's gym.

So, I want to make a bot that can move on the chess table like the knight. And it is given a reward each time it moves and does not leave the table or step in an already visited place. So, it gets better rewards if it survives more.

Or there is a better approach to this problem? Also, I would like to display the best knight in each generation.

I'm not very advanced at reinforcement learning (I'm still studying it), but this project really caught my attention. I know well machine learning and deep learning.

Do I need to start implementing a new OpenAI's gym environment and start all from scratch, or there is a better idea?

Model your problem as an MDP

To solve a problem with reinforcement learning, you need to model your problem as a Markov decision process (MDP), so you need to define

the state space,
the action space, and
the reward function

of the MDP.

Understand your problem and the goal

To do define these, you need to understand your problem and define it as a goal-oriented problem.

In the knight tour problem, there's a knight that needs to visit each square of a chessboard exactly once. The knight can perform only $L$-shaped moves (as for the rules of chess). See the animation below (taken from the related Wikipedia article).

The goal is then, by making $L$ moves, to find a path through the squares such that each square is visited exactly once.

What is the state space?

You could think that the state space $S$ could be the set of all squares of the chessboard. So, if you have an $n \times n$ chessboard, then $|S| = n^2$, i.e. you will have $n^2$ states.

However, this can be problematic because a square alone doesn't tell you all the information that you need to know to take the optimal action. So, you need to define the states such that all available information is available to the agent, i.e. you need to define a state as the position of the current square and the position of the other available squares.

What is the action space?

The action space could be defined as the set of all actions that the knight can take across all states. Given that the knight can only take $L$ moves, whenever the knight is in state $s$, only $L$-shaped actions are available. Of course, it is possible that, for each state $s$, there's more thane one valid $L$-shaped action. That's fine. However, the chosen $L$-shaped action will definitely affect your next actions, so we need a way of guiding the knight. That's the purpose of the reward function!

What is the reward function?

The reward function is typically the most crucial function that you need to define when modeling your problem as an MDP that needs to be solved with an RL algorithm.

In this case, you could give a reward of e.g. $1$ for each found path. More precisely, you will let your RL agent explore the environment. If it eventually finds a correct path (or solution), you will give it $1$. You could also penalise the knight if it ends in a situation where it cannot take an $L$-shaped action anymore. Given that you don't really want this to happen, you could give it a very small reward e.g. $-100$. Finally, you could give it a reward of $0$ for each action taken, which could imply that you don't really care about the actions that the knight takes, as long as it reaches the goal, i.e. find a path through the chessboard.

The design of the reward function will highly affect the behaviour and performance of your RL agent. The above-suggested reward function may actually not work well, so you may need to try different reward functions to get some satisfactory results.

Which RL algorithm to use?

Of course, you will also need to choose an RL algorithm to solve this problem numerically. The most common one is Q-learning. You can find its pseudocode here.

How to implement this with OpenAI's gym?

You probably need to create a custom environment and define the state and action spaces, as well as the reward function. I cannot tell you the details, but I think you can figure them out.

Is RL the right approach to solve this problem?

RL isn't probably the most efficient approach to solve this problem. There are probably more efficient solutions. For example, there's a divide-and-conquer approach, which I am not familiar with, but that you may also try to use and compare with the RL approach.

You could also read the paper Solution of the knight's Hamiltonian path problem on chessboards (1994), especially, if you are already familiar with the Hamiltonian path problem (HPP). Note that the knight tour problem is an instance of the HPP.


Unix timestamps for Recurrent Neural Networks I want to use RNN for classifying whole sequences of events, generated by website visitors. Each event has some categorical properties and a Unix timestamp:
sequence1 = [{'timestamp': 1597501183, 'some_field': 'A'}, {'timestamp': 1597681183, 'some_field': 'B'}]
sequence2 = [{'timestamp': 1596298782, 'some_field': 'B'}]
sequence3 = [{'timestamp': 1596644362, 'some_field': 'A'}, {'timestamp': 1596647951, 'some_field': 'C'}]

Unfortunately, they can't be treated as classic time series, because they're of variable length and irregular, so timestamps contain essential information and cannot be ignored. While categorical features can be one-hot encoded or made into embeddings, I'm not sure what to do with the timestamps. It doesn't look like a good idea to use them raw. I've come up with two options so far:

Subtract the minimum timestamp from every timestamp in the sequence, so that all sequences start at 0. But in this case the numbers can still be high, because the sequences run over a month.
Use offsets from previous event instead of absolute timestamps.

I'm wondering if there are common ways to deal with this? I haven't found much on this subject.

I would say offsets from previous N-event is necessary in this case. You can also encode their max and avg as other features in order to compress the irregular sampling times into a vector.


Extract Features at Multiple Image-Scales I try to replicate the results of this paper. They state, that they used VGG16- and VGG19-models pretrained on imagenet and used the output of the last convolutional layer (without relu and max-pooling) as feature vectors.
To configure the model accordingly i do:
from tensorflow.keras.models import Model
from tensorflow.keras.applications import VGG16

base_model = VGG16(include_top=False)  # Cut off the fully-connected-layers
model = Model(inputs=base_model.input, outputs=base_model.get_layer('block5_conv3').output)  # Discard the last max-pooling-layer
model.layers[-1].activation = None  # Change activation-function of last layer from Relu to Linear

and i get a feature-tensor of shape (1, 14, 14, 512) for the default input-shape of (224, 224, 3). So far, this looks the way i would want it to be. However, the authors state:

The VGG-M [equivalent to VGG16 in the context of the paper] convolutional features are extracted as the output of the last  convolutional layer, directly from the linear filters excluding ReLU  and max pooling, which yields a field of 512-dimensional descriptor  vectors

Now, i have stated above, that the number 512 is part of my output-shape. However i thought, that it means that i get back 512 individual image-patches of size 14x14! The only way i could think of, that would get me 512-dimensional descriptor vectors would be something like this:
features = model.predict(img)

feature_vectors = []
    for i in range(features.shape[1]):
        for j in range(features.shape[2]):
            feature_vectors.append(features[0, i, j, :])
feature_vectors = np.array(feature_vectors)

But then i would slice through all of the existing image-patches!
Question 1:
Did the authors mean to do just that? Is this a common practice anyone her has used before? All the tutorials or blogposts i found online just flatten() the output-tensor and add it to the database of existing feature-vectors.
Question 2:
The authors also state, that:

...local descriptors are extracted at multiple scales, obtained by rescaling the image by factors $2^s, s=−3,−2.5,\dots,1.5$ (but, for efficiency, discarding scales that would make the image larger than $1024^2$ pixels).

I can totally extract images at different scales, by specifying the input_shape, when instantiating the VGG16-model. My method of getting 512-dimensional feature-vectors stated above would also work in that case, even though the output-tensor could be much larger (i.e. a shape of (1, 64, 64, 512)) in case of a bigger image.
Is this the way to do feature-extraction at multiple scales?

Not sure I have understood well your second question but I am gonna try to see if I can help you.
Question 1
Yes, the authors meant just that. I see were the confusion might come. So:

The authors say: "[...] yields a field of 512-dimensional descriptor vectors"
You say: "[...] i get back 512 individual image-patches of size 14x14"
You get: "i get a feature-tensor of shape (1, 14, 14, 512)"

So when the authors say a field of K-dimensional vector, with the word field they implicitly say that is a 2D vector (so $[H,W]$) of K dimensions, so $[H,W,K]$ which is exactly what your are getting (only with the batch dimension first, that Keras ignores in its input, that's why I prefer pytorch, it is more explicit).
So being explicit, from an input image $I:[B,H,W,C]=[1, 224, 224, 3]$ you get a field (or patch, or feature) vector, a 2D vector, of $K=512$ dimensions (or features, or descriptors) so $F:[B,H_f, W_f, K]=[1,14, 14, 512]$
What you are doing in your snippet is taking 1D vectors of $K=512$ dimensions, so you are extracting K-dimensional vectors from a K-dimensional feature map (or patch or field).
The flatten() goes in the second question.
Question 2
There are a lot of ways of doing feature-extraction at multiple scales. Say for example you extract features at 3 levels so you get:

$P_1:[H_1, W_1, C_1]=[56, 56, 100]$
$P_2:[H_2, W_2, C_2]=[26, 26, 200]$
$P_3:[H_3, W_3, C_3]=[13, 13, 300]$

This vectors are not concatenable. You indeed could flatten() them over the $[H,W]$ dimensions and concat over the channel dimension. But you would loose the 2D spatial relationship of images. What is normally done (from FPN, Retinanet EfficientNet, EfficientDet...) is the following:

You extract the previous feature maps at different scales
You process them over a FCN head of your network (so last dimension is bounding box coordinates or class probabilities)
You concatenate over last dimension

After extracting $P_1, P_2, P_3$ and passing by a FCN classification head on your network you would get (given you have 10 classes):

$P_1':[H_1, W_1, C_1]=[56, 56, 10]$
$P_2':[H_2, W_2, C_2]=[26, 26, 10]$
$P_3':[H_3, W_3, C_3]=[13, 13, 10]$

Now you have a classification vector per each "pixel" on each feature dimension. So now it makes more sense to flatten them to get 1D vectors of 10 class probs:

$P_{concat}=[N, 10]$ where $N$ is the flattened number of dimensions from $H,W$ of each scale feature map

If you want the implementation details go to RetinaNet in papers with code. I would link one very nice implementation but it is in pytorch, and I see you use Keras.


What was the first machine that was able to carry on a conversation? What was the first AI that was able to carry on a conversation, with real responses, such as in the famous 'I am not a robot. I am a unicorn' case?

A 'real response' constitutes a sort-of personalized answer to a specific input by a user.

In 1986, the first PC therapist program was written by Joseph Weintraub. This program won the first Loebner Prize in 1991, and then again in 1992, 1993 and 1995.

In 1981 or 1982, Jabberwacky was founded, which is the foundation of the current Cleverbot. Jabberwacky  appeared on the internet in 1997, reaching the third place for the Loebner Prize in 2003, the second place in 2004, and won in 2005 and 2006. In 2008, Cleverbot was launched as an variant of Jabberwacky.

I'm not sure these are really the earliest, but that also depends on what you want earliest (programming started, first conversation,  first decent conversation, etc.). Also, it depends on what you call a "real response".


Does a better discriminator in GANs mean better sample generation by the generator? Since the discriminator defines how the generator is updated, then building a discriminator with a higher number of parameters/more layers should lead to a better quality of generated samples. So, assuming that it won't lead to overwhelming the generator (discriminator loss toward 0) or mode collapse, when engineering a GAN, I should build a discriminator as good as possible?

If you start to train a GAN in a way in which the discriminator becomes very powerful than the generator, then the training of generator will not be very much successful. In the same way, if the discriminator is too low in comparison with generator, it would let it produce any thing. So, either way, it will affect the training of the entire system.
So, if you see here, both generator and discriminator and competing against each other and on the other side they are dependent on each other for efficient training.
But I've read in a research paper that adding noise to discriminator is good for the overall stability. And sometimes people also change the labelling system as well. For instance, the value for YES is 1 and people change it to 0.9. This actually keeps the discriminator from being over confident.


Neural network with logical hidden layer - how to train it? Is it policy gradient problem? Chaining NNs? I am doing neural machine translation task from language S to language T via interlingua L. So - there is the structure:

S ->
encoding of S (crisp) ->
S-L encoder -> S-L decoder ->
encoding of L (non-crisp, coming from decoder) ->
L ->
encoding of L (crisp) ->
L-T encoder -> L-T decoder ->
encoding of T (non-crisp, coming from decoder) ->
T

All of this can be implemented in Pytorch more or less adapting the usual encoder-decoder NMT. So, the layer of interlingua L acts as a somehow symbolic/discrete layer inside the whole S-L-T neural network. My question is - how such system can be trained in end-to-end (S-T) manner? The gradient propagates from the T to the L and at the L one should do some kind of symbolic gradient? I. e. one should be able do compute the difference L1-L2?

I am somehow confused by such setting. My question is - is there similar networks which contain the symbolic representation as the intermediate layer and how one can train such system. I have heard about policy gradients but are they relevant to my setting?

Essentially - if I denote some neural network by symbols x(Wi)y, then the training of this network means, that I change Wi and x stays intact. I.e. the last member of backpropagation equation has the form d.../dw1. But if I combine (chain!) 2 neural networks x(Wi)y-y(Wj)z, then the the last backpropagation term for the y(Wj)z has the form (d.../dw1+d.../dy) and hence both the w1 and y should be changed/updated by the gradient descent too. So, doesn't some ambiguity arise here? Is such chaining of neural networks possible? Is is possible to train end-to-end chains of neural networks?

I am also thinking about use of evolutionary training.

The chain rule applies here as usual, and the term symbolic gradient is interesting.

How such might apply will depend much on the nature of the policy layer representation and the connections of that layer or layers to other non-policy layers.

A binary threshold layer may feed a logical inference layer.
A sigmoid function might feed a fuzzy logic layer.
The logical layer could be a production system adapted to look like a layer.
The logical layer might be more like a PLD (programmable logic device).
The output of the logical layer might be mapped one to one with a perceptron or an LSTM layer

In the example in the question, we can't, from the information given in the question, assume that discrete means a discrete approximation of a curve, since the term used is discrete/symbolic. Since the nature of logic cannot be generalized in a differentiable closed form, it may be that it has to be probabilistically represented in closed form so a derivative may be obtained, if that is possible in the specific case under study.

Back propagation is a feedback mechanism to provide corrective signaling to the portions of the system that must be corrected. The hope is that the direction of the correction during back propagation leads to convergence on the global minimum of loss without severe delay, which is not always easy to make happen, thus Gaussian noise injection explicitly or indirectly through mini-batching, momentum, multiple initialization states, parallel searches with different hyper-parameters, and other techniques.

One technique used quite a bit in analog systems and entering use in digital systems in recent years is the idea of multiple corrective mechanisms. Control theory for instrumentation and countermeasure aeronautics is thick with this concept. For the case given in this question, it can look like this, with $f$ representing feed forward layer with some activation function, $\ell$ representing some logical inference container, $p$ is the parameter set for the function of the same subscript, $\epsilon$ is the evaluation function (error, loss, value, benefit), $b$ representing the corrective feedback using the back propagation, the Jacobian, and the chain rule.

$$ f_1 \quad \rightarrow \;\quad \ell \quad \rightarrow \quad\ f_2 \quad \rightarrow \quad\ \epsilon \\
 p_1 \leftarrow \, b \leftarrow \ell\, ; \quad\quad\quad \; p_2 \, \leftarrow b \leftarrow \epsilon  \\
\quad\quad\quad\quad\quad \ell \quad\quad \leftarrow \quad\quad\quad\quad\quad \epsilon
 $$

In the last line $\epsilon$ is simply a fact (piece of information) passed into the a rules engine session or a floating point representation of a feedback signal assigned as a probability to a rule in a fuzzy logical container. In the first half of the second line, the logical unit is responsible for providing the evaluation function output.

In Norbert Wiener's Cybernetics, he states the following1 with regard to automating the steering of ships via rudder control, from which cybernetics got its name.

  In the important book of MacColl2, we have an example of a complicated system which can be stabilized by two feedbacks but not by one.

He then procures the differential equations and proves MacColl's conclusion. That conclusion is used in nearly every instrumentation amplifier circuit on a chip sold today, if not all of them. There is no way to get the desired stability without the multiple feedback paths.

Similarly, it may not be possible to stabilize the convergence of $f_1, \ell, and f_2$ via a single mechanism of back-propagation using $\epsilon$ and the Jacobians to descend a gradient.

References

[1] Norbert Wiener's *Cybernetics, 1948, MIT, p 106 in the 1996 printing

[2] LeRoy A. MacColl. Fundamental Theory of Servomechanisms. Van Nostrand, New York, 1945


How do I solve this optimal control problem with reinforcement learning? I am new to reinforcement learning. I would like to solve an optimal control problem with reinforcement learning.

The objective is for a wolf to catch a rabbit. The wolf and the rabbit run on a plane. The time is discrete. At every time step, the wolf can only run straight, change direction by 10 degrees to the right or left, change the speed by 0.1 m/s or remain at the same speed. It starts running in some random direction, and then sees the rabbit and starts chasing it. For the time being, let's assume that the rabbit sits still.

It looks like this problem is a continuous state space and discrete action space.

I have tried to use DQN in Keras, but I am not sure that I am using correct state variables/reward. Currently, the state variables are the velocity vector of the wolf, the distance vector from the wolf to the rabbit. The reward at each time point is the negative current time. When the wolf catches the rabbit, the reward is 1000 - current time (the wolf is penalized for running too long).

Can somebody provide me some guidance? Eventually, I would add brains to the rabbit so that it tries to escape the wolf and compare to the optimal control solution.

I have tried to use DQN in Keras, but I am not sure that I am using correct state variables/reward.

You have a wide range of choices that are all valid. As it is a simple control and learning scenario, provided you cover basics (described in a moment), then the difference in your choices are about how easy you make it for the agent to learn. You may actually want to set things up so that there is a certain difficultly level, so you get to compare different agent designs. You don't necessarily need the "best" state representation and reward function designs here. Just some that work.

State representation

The most important factor is that your state has the Markov property - that it contains enough information that the agent really can predict expected future rewards.

It is OK for there to be randomness, but generally not OK for there to be important hidden variables that affect the outcome in a major way. If there are such variables in the environment, but not in the state representation, then a simple agent like DQN will struggle.

Reward function

The reward function should capture the end goals that you are interested in, and ideally as simply and purely as possible given that. The purpose of most RL methods is to maximise the expected sum of reward, and they are built to deal with sparsity - you don't need to help or signal the same thing twice or give rewards for "getting close" etc, although sometimes this can improve learning speed if done well.

I have also written a more comprehensive answer about reward function design as an answer to Deciding on a reward per each action in a given state (Q-learning)

  Currently, the state variables are the velocity vector of the wolf, the distance vector from the wolf to the rabbit.

This seems reasonable. Do consider that for a neural network you will want to keep the scale of these to within a nice range e.g. -1 to +1 for all features. So you may want to scale from whatever you are using for the values in the environment.

You can make this problem easier to learn by using polar co-ordinates. That removes the need for the neural network to solve the inherent trigonometry problem when deciding if turning left or right is better. I have solved a very similar pursuit RL problem and compared between cartesian and polar co-ordinates, and the difference is very large between the two representations for a DQN based agent.

Specifically for the easiest learning you want the difference vector between your agent's current heading and the target, expressed as a difference in heading plus distance to go. If you do this, sometimes the initial randomised neural network will already be efficient at solving the problem.

However, it may be interesting for you to see how well different agents do solve that trigonometry problem of converting a difference in vector coordinates and velocity, into a direction to turn. You could even make it harder and give absolute location coordinates for both agent and the target, plus the agent velocity (3 vectors instead of 2), requiring the neural network to approximate even more complex maths before it solves the problem.

  The reward at each time point is the negative current time. When the wolf catches the rabbit, the reward is 1000 - current time (the wolf is penalized for running too long).

That should work, but seems more complex than you need. You are already penalising the wolf for each time step during the chase, so there is no real need to have any extra formula for the final reward. In fact you can just have a final reward of 0 for reaching a satisfactory goal (providing there are no unsatisfactory end points that you want it to avoid - perhaps yo umay make the environment trickier later on?). The agent will simply try to resolve the episode quickly because that will still maximise the (negative total) return.

An alternative might be to reward some value for catching the rabbit e.g. +10, and zero on each time step. Then to encourage a fast resolution you would need to use discounting, so that the agent values being nearer to the rabbit higher than being further away because the potential reward is fewer time steps away.

There is also no need to use large values. Values are relative, so if you have some minor issue to solve plus a main goal, sometimes it is worth having a wide range of values. Here you don't need it. Having a value of 1000 may challenge your neural network to learn properly (because error for the first time the wolf catches a rabbit would be so large it may cause a large step in weights - enough to destabilise learning in the NN), for no real benefit.

For your first reward scheme I recommend a fixed reward of e.g. -1 per time step, and to end the episode when the wolf catches the rabbit. No multiplications or offsets.

If you have a very large area for the wolf to explore compared to the distance that the wolf needs to be from the rabbit in order to catch it, it may help to give some reward for proximity to the rabbit. The simplest to implement and learn would just be the difference in distance between the wolf and rabbit between $t$ and $t+1$ multiplied some small factor to put this into a similar range to the penalty per time step. Note if you extend this reward out to any significant distance that it should make the learning problem far easier (combine polar coordinates and this reward system will make it very easy for the DQN to learn - it should take only a few episodes to get near optimal behaviour, if your learning hyperparameters are good for the DQN).

Without any proximity reward, you will rely on the wolf literally bumping into the rabbit through random behaviour, before it will have any data example that getting the vector between itself and the rabbit close to (0,0) is a good thing. You may need to have a relatively large capture radius, plus limit the area that the wolf (and eventually rabbit) can explore, in order to avoid very long sequences of random behaviour where nothing is learned initially.


Which parsing algorithm can I use for NLP question answering system? I am currently working on my last project before graduating. For this project, I have to develop a Natural Language Question Answering System. Now, I have read quite some research papers regarding this topic and have figured out everything except for the parsing algorithm.
The NL Q-A will be programmed in Python, and I will use the spaCy library to finish this project. However, I am stuck when it comes to parsing algorithms. I managed to reduce the parsing algorithms to 3:

Cocke-Kasami-Younger (CKY) algorithm
Earley algorithm
Chart Parsing algorithm

Note: I know that all three algorithms are chart parsing algorithms. I also know that the Earley algorithm is context-free, but has a low efficiency for a compiler.
What I don't know is: Which one should I pick? (Please, provide a non-subjective answer to this question!)
The system is for a specific domain. The answer to the natural question will be displayed in the form of the result of a calculation of some kind. Preferably in the tabular or graphical form.
I have done my research. However, I probably do not understand the algorithms properly, which makes it difficult to make a selection.The algorithm should be efficient and perhaps outperform others.

I have been reading and reading, and found answers to almost all my questions.
I am sticking to Earley algorithm, given that it offers a dynamic programming approach (CKY does the same). Both algorithms are chart parsing algorithms.
Earley is a context-free, top-down parsing algorithm, which makes it a goal-driven algorithm. From start symbol down. Furthermore, it is more efficient than the CKY algorithm.
Here are the slides that provide more info.