How to determine if an Amazon review is likely to be fake using text classification I'm currently in the research stage of building a web app in ASP.NET where the user can input a URL to an Amazon product, then the app would determine how likely its reviews are to be genuine. I need help figuring out what algorithm to use in determining if a certain review is likely to be deceptive. I want my app to behave similarly to Fakespot or ReviewMeta. I realise a tool like this won't be 100% accurate and that's fine.

So far I read parts of this book which seems to be recommended a lot to NLP newbies but haven't found anything that applies to such a specific problem. I also read this article but it's based on hotel, restaurant and doctor reviews. I'm trying to find a more general method that can be applied to any product. Any help would be greatly appreciated!

This will not be that hard of a problem once you have a lot of training data. But, before you have a lot of training data, you will need to get some training data one way or another. You will need a lot of training data for quite a few of the models that will give you a high accuracy. Then, you will probably want to use a Long short term memory recurrent neural network along with a word2vec model, or maybe even a sentence/ paraphraph2vec model. This will be able to give you some fairly good results with a little bit of tweaking, but if you want really accurate results you will want to try an ensemble. An ensemble is when you use multiple neural networks that provide an output that gets fed through another classifier(usually XGboost) in order to achieve better classification results. You might also want to try a little bit of feature engineering. Finally, in order to implement this model, you can take one of several approaches. First off, you can use tensorflow serving. Tensorflow was created by google and used by much of the machine learning community. However, tensorflow was written in python, so you will need to use tensorflow serving in order to use it within your Asp.net app. Or, you could use microsofts CNTK. Or finally, you could use a custom built implementation, but this would take a lot of time and not be worthwhile unless you are doing research. Also, if you really do not want to use deep learning for one reason or another, you can probably use a simpler model, but you will need to understand your data. You will need also need to understand your data if you are doing deep learning but maybe not as much.


In reinforcement learning, is it possible to make some actions more likely? In a general DQN framework, if I have an idea of some actions being better than some other actions, is it possible to make the agent select the better actions more often ?

For single-step Q learning, the behaviour policy can be any stochastic policy without any further adjustment to the update rules.
You don't have to use $\epsilon$-greedy based on current Q function approximation, although that is a common choice because it works well in general cases. However, you should always allow some chance of taking all actions if you want the algorithm to converge - if you fixed things so that bad actions were never taken, the agent would never learn that they had low value.
Probably the simplest way to use your initial idea of best actions is to write a function that returns your assessment of which action to take, and use that with some probability in preference to a completely random choice. At some point you will also want to stop referencing the helper function (unles it is guaranteed perfect) and use some form of standard $\epsilon$-greedy based on current Q values.
I have done similar with a DQN learning to play Connect 4, where the agent would use a look-ahead search function that could see e.g. 7 steps ahead. If that was inconclusive it would use argmax of current Q values. Both these fixed action choices could be replaced, with probability $\epsilon$, with a random action choice to ensure exploration. It worked very well. You could replace the look-ahead search in my example with any function that returned "best" actions for any reason.
There are some other ways you can skew action selection towards better looking action choices. You could look into Boltzmann exploration or upper confidence bounds (UCB) as other ways to create behaviour policies for DQN.


Why does KL divergence not satisfy the triangle inequality? $D_{KL}=\sum_i p(x_i)log(p(x_i)/q(x_i)$

Also can't you make it satisfy the triangle inequality by taking the absolute value of the information at every point?

To prove that the KL divergence does not satisfy the triangle inequality, you just need a counterexample.

Definitions

KL divergence

Let's first recapitulate the definition of KL divergence for discrete probability distributions $p$ and $q$ (for simplicity).

$$
D_{\text{KL}}(p\parallel q)
=
\sum_{x\in {\mathcal {X}}}
p(x)\log
\left(
\frac {p(x)}{q(x)}
\right)
$$

Triangle inequality

Let's also recap the definition of the triangle inequality for distance $d$, which can be expressed as

$$
d(p,q) \leq d(p, r) + d(r, q),
$$
where $p, q$ and $r$ are probability distributions.

Proof

For simplicity, let the sample space be $\mathcal{X} = \{0, 1 \}$. Consider now the following three discrete probability distributions.

$$
p(x)=
\begin{cases}
\frac{1}{2}& \text{if } x = 0\\
\frac{1}{2}& \text{if } x = 1
\end{cases}
$$

$$
r(x)=
\begin{cases}
\frac{1}{4}& \text{if } x = 0\\
\frac{3}{4}& \text{if } x = 1
\end{cases}
$$

$$
q(x)=
\begin{cases}
\frac{1}{10}& \text{if } x = 0\\
\frac{9}{10}& \text{if } x = 1
\end{cases}
$$

Let's now apply the definition of the triangle inequality and the KL divergence for these discrete probability distributions. So, we want to show the following inequality does not hold.

$$
D_{\text{KL}}(p\parallel q)
\leq
D_{\text{KL}}(p\parallel r)
+
D_{\text{KL}}(r\parallel q)
$$

which can be expanded to

\begin{align}
\sum_{x\in \{0, 1 \}}
p(x)\log
\left(
\frac {p(x)}{q(x)}
\right)
&\leq
\sum_{x\in \{0, 1 \} }
p(x)\log
\left(
\frac {p(x)}{r(x)}
\right)
+
\sum_{x\in \{0, 1 \}}
r(x)\log
\left(
\frac {r(x)}{q(x)}
\right)
\end{align}

which roughly corresponds to

\begin{align}
0.51 &\leq 0.14 + 0.09 \\
0.51 &\leq 0.24
\end{align}
which is clearly false.

Given that the triangle inequality does not hold in one case, it doesn't hold in all cases, so the triangle inequality does not hold for the KL divergence.

$$\tag*{$\blacksquare$}$$

For reproducibility, I have used the following Python (3.7) program to compute the KL divergences.

from math import log

one = (1 / 2 * log((1 / 2) / (1 / 10))) + (1 / 2 * log((1 / 2) / (9 / 10)))
two = (1 / 2 * log((1 / 2) / (1 / 4))) + (1 / 2 * log((1 / 2) / (3 / 4)))
three = (1 / 4 * log((1 / 4) / (1 / 10))) + (3 / 4 * log((3 / 4) / (9 / 10)))

print(one)
print(two)
print(three)
print(two + three)
print(one <= two + three)


Why is my Keras prediction always close to 100% for one image class? I am using Keras (on top of TF 2.3) to train an image classifier. In some cases I have more than two classes, but often there are just two classes (either "good" or "bad"). I am using the tensorflow.keras.applications.VGG16 class as base model with a custom classifier on top, like this:
input_layer = layers.Input(shape=(self.image_size, self.image_size, 3), name="model_input")
base_model = VGG16(weights="imagenet", include_top=False, input_tensor=input_layer)
model_head = base_model.output
model_head = layers.AveragePooling2D(pool_size=(4, 4))(model_head)
model_head = layers.Flatten()(model_head)
model_head = layers.Dense(256, activation="relu")(model_head)
model_head = layers.Dropout(0.5)(model_head)
model_head = layers.Dense(len(self.image_classes), activation="softmax")(model_head)

As you can see in the last (output) layer I am using a softmax activation function. Then I compile the whole model with the categorical_crossentropy loss function and train with one-hot-encoded image data (labels).
All in all the model performs quite well, I am happy with the results, I achieve over 99% test and validation accuracy with our data set. There is one thing I don't understand though:
When I call predict() on the Keras model and look at the prediction results, then these are always either 0 or 1 (or at least very, very close to that, like 0.000001 and 0.999999). So my classifier seems to be quite sure whether an image belongs to either class "good" or "bad" (for example, if I am using only two classes). I was under the assumption, however, that usually these predictions are not that clear, more in terms of "the model thinks with a probability of 80% that this image belongs to class A" - but as said in my case it's always 100% sure.
Any ideas why this might be the case?

Traditional neural networks can be over-confident (i.e. give a probability close to $0$ or $1$) even when they are wrong, so you should not interpret the probability that it produces as a measure of uncertainty (i.e. as a measure of how much it is confident that the associated predicted class is the correct one), as that it is essentially wrong. See this and this answers for more details about this.
Given that this overconfidence is not desirable in many scenarios (such as healthcare, where doctors also want to know how confident the model is about its predictions, in order to decide whether to give a certain medication to the patient or not), the ML community has been trying to incorporate uncertainty quantification/estimation in neural networks. If you are interested in this topic, you could read the paper Weight Uncertainty in Neural Network (2015) by Blundell et al., which proposes a specific type of Bayesian neural network, i.e. a neural network that models the uncertainty over the actual values of the weights, from which we may also quantify/estimate the uncertainty about the inputs. This paper should not be too difficult to read if you are already familiar with the details of variational-autoencoders.
So, the answer to your question is: yes, it's possible that the output probability is close to $1$ because neural networks can be over-confident. (I am assuming that the values returned by tf.keras's predict method are probabilities: I don't remember anymore, so I assumed that you did not make any mistake).
A similar question was already asked in the past here. The accepted answer should provide more details about different types of uncertainty and solutions.


What is the theoretical basis for the use of Cross Validation set? So let's follow this line of reasoning. We use a MLE estimator (implementation doesn't matter) and we have a train set. We assume that we have sampled training set from a Gaussian distribution $\mathcal N(\mu, \sigma^2)$.

Now we split the dataset into training, validation and test sets. The result will be that each will have maximum likelihoods for the following Gaussian distributions $\mathcal N(\mu_{train}, \sigma^2_{train}), \mathcal N(\mu_{cv}, \sigma^2_{cv}), \mathcal N(\mu_{test}, \sigma^2_{test})$.

Now let's assume the  case where $\mu_{cv}<\mu_{train}<\mu_{test}$ and $\mu_{train}<\mu<\mu_{test}$. Clearly, if we perform validation using this split, then the model which gets selected, will be closer to  $\mu_{cv}$ which will worsen the performance on actual data, whereas if we only used train set the performance could actually be better (this is the simplest case without taking into account the effect of variance).

So, like this we will have a $4!$ combinations between the means and each one might improve or worsen the performance (probably in $50 \%$ cases performance will be worsened, assuming symmetry).

So what am I missing here? Was my aforementioned assumptions wrong? Or does the validation set has a completely different purpose?

I think Cross-Validation serves a completely different purpose.

From your post, it looks like you think we would use CV to get a better estimate of the parameters of our model (i.e. the model parameters after cross validation are closer to the parameters of the test data).

In fact, we use CV to get an estimate of generalization error while keeping our test set outside the training process. That is, we use it to answer the question "What is the size of the difference between my training and testing performance likely to be?". If you have an estimate of this that you are confident in, you can be confident that when you deploy a model to your customers, the model will actually work as you expect.

If you're only going to build a single model, then you don't need cross validation. You just train the model on the training data, and test it on the test data. Then you have an unbiased estimate of generalization error.

However, we might want to try out many different kinds of models, and many different parameters (broadly, we might want to do hyperparameter tuning). To do this, we need to understand how generalization error changes as we change our hyperparameters, and then use this information to pick hyperparameter values that we think will minimize the actual error when we deploy the model.

You could do this by training different models on the training set, and then testing them on the test set, recording the difference in model performance on the two sets. If you use this as a basis to pick a model though, you have effectively pulled the test set inside your training process (model parameters were implicitly selected using the test set, since you picked the parameters with the lowest test error). This bias will make your true generalization error much larger than what you observed.

As a stop gap, you could split your training set into a 'real training' set and a validation set. You could train models on the 'real training' set, and then measure their performance on the validation set. The difference would be a biased (but hopefully still useful) estimate of generalization error. You could then test against the test set just once (at the end) to get an unbiased estimate that you can use to decide whether or not to deploy the model.

A better workflow is to use CV on the training set to get an estimate of generalization error during hyperparameter optimization. You get K samples for k-fold cross-validation, so you can do statistical testing to see whether one model truly has better generalization error than another, or whether it's just a fluke. This decreases the degree of bias in your estimates of generalization error. Then, once you've completed hyperparameter optimization, you can run your final model against the test set once to obtain a truly unbiased estimate of your final generalization error.


Why is an expectation used instead of simple sum in GANs? Why do the GAN's loss functions use an expectation (sum + division) instead of a simple sum?

I'm assuming you mean the original loss explained in the original GAN paper. This practice was by design and for interpretation. An expectation is what your expecting to get, which is generally a good objective when your are sampling. Note this isn't just in GANs, it's in most objective functions used in a widespread of problems.
In general practice, though, dividing is good because it works as a normalization. Let's say you train one batch with 10 elements and the next with 8. Wouldn't you want each training example to be weighted equally?
Also, even if you use equal-sized batches, then the "division" you mention gets eaten up by the learning rate if you're using some form of gradient scheme.


What are the learning limitations of neural networks trained with backpropagation? In 1969, Seymour Papert and Marvin Minsky showed that Perceptrons could not learn the XOR function.
This was solved by the backpropagation network with at least one hidden layer. This type of network can learn the XOR function.
I believe I was once taught that every problem that could be learned by a backpropagation neural network with multiple hidden layers, could also be learned by a backpropagation neural network with a single hidden layer. (Although possibly a nonlinear activation function was required).
However, it is unclear to me what the limits are to backpropagation neural networks themselves. Which patterns cannot be learned by a neural network trained with gradient descent and backpropagation?

Multilayer Perceptron (MLP) can theoretically approximate any bounded, continuous function. There's no guarantee for a discontinuous function. There are plenty of important discontinuous functions, like, say, the prime counting function.
The prime counting function $\pi(n)$ is simply equal to the number of primes less than or equal to $n$. It has a discontinuity about each prime $p$, so good luck trying to approximate this with a neural network!
However, this function is extensively studied and extremely important in number theory. See the Riemann hypothesis.


What is the name of an AI whose primary goal is to create a better AI? A general AI x creates another AI y which is better than x.

y creates an AI better than itself.

And so on, with each generation's primary goal to create a better AI.

Is there a name for this.

By better, I mean survivability, ability to solve new problems, enhance human life physically and mentally, and advance our civilization to an intergalactic civilization to name a few.

I don't think there is a single standard word or phrase that covers just this concept. Perhaps recursive self-improvement matches the idea concisely - but that is not specific AI jargon.

Very little is understood about what strength this effect can have or what the limits are. Will 10 generations of self-improvement lead to a machine that is 10% better, 10 times better, or $2^{10}$ times better? And by what measure?

Some futurologists suggest this might be a very strong effect, and use the term Singularity to capture the idea that intelligence growth through recursive self-improvement will be strong, exceed human intelligence, and lead to some form of super-intelligent machine - the point at which this goal is reached is called The Singularity. Ray Kurzweil is a well-known proponent of this idea.

Specifically, use of the term Singularity implies more than just the basic recursion that you suggest, and includes assumptions of a very large effect. Plus technically, it refers to a stage that results from the recursion, not the recursion itself.

However, despite the popularity of it as a concept, whether or not such self-improving system will have a large impact on the generation of intelligent machines is completely unknown at this stage. Related research about general intelligence is still in its infancy, so it is not even clear what would count as being the first example system x.


What is the fundamental difference between neural networks for classifying and generating data What is the fundamental difference between NN for classifying data and generating data?

Most examples show how neural networks can be used to classify data. Like is it an image of a dog or a cat. However, there are applications where NN are used to create images and even write short stories.

The formal name for this difference is "generative" vs "discriminative" models.
By default, a supervised learning process using a simple feed-forward neural network and a set of training data with expected answers will produce a discriminative model. It is hard to use such a model to generate content directly.
Differences between discriminative and generative models tend to be at the architecture level or deeper, although a few generative techniques are simple variants or re-purposing of discriminative networks. There is no single "fundamental" difference in terms of NN design. About the only common theme is that generative models are more complex and harder to work with than discriminative ones.
Images

Probably the most popular currently for image generation are Generative Adversarial Networks or GANs. These train a Generator to create an image from random inputs, alongside a Discriminator that tries to spot fakes compared to real images. Training them together results in a generator network that gets progressively better at creating fake data.

Also suitable for image generation are Variational Autoencoders (VAEs) which learn essentially by compressing a set of images down into a small representation, and as a result become able to "uncompress" similar representations.

There are also VAEGANs, which combine VAEs and GANs

There are other models which generate examples from a dataset. For example, Restricted Boltzmann Machines (RBMs) are similar to neural networks, but the neurons fire randomly, and RBMs require a different training process to NNs.

GANs, VAEs, VAEGANs, RBMs can be used to generate data in any simple non-sequential datasets, they are not restricted to just images, but recent work with GANs for example has excelled there.

For a different kind of generator, you can look at how Deep Dream works. The interesting thing here is that it is a modification of a feed forward network that has been trained by supervised learning. Essentially to run Deep Dream, you take an existing image and change it so that it maximises some internal part(s) of the existing neural network, "training" the image as if it were a set of weights.

The difference between Deep Dream and other generative techniques is that it does not generate content that could be in the training set - it is no good for creating "realistic" images.

Text

Recurrent Neural Networks, and specifically Long Short-Term Memory (LSTM) networks can be used to model word sequences by training them to predict the next word. Because the resulting model is probabilistic (e.g. chance of next work being "Hello" is ~0.001%) then a simple method to make this generative is to sample from the predictions randomly, and then feeding them back into the sequence to see what it will predict next. There is a great blog post about this technique by Andrei Karpathy.

Audio and Music

DeepMind's WaveNet architecture is similar to a CNN predicting the next samples of audio from a given input. It can be used in speech or music generation.

LSTMs are a popular choice here, as well as for language models.

One important caveat. Nearly all these models are hard to understand, work with and train, compared to simpler supervised techniques. Probably the easiest to get to grips with if you want to understand them well enough to implement your own versions, are Deep Dream, VAEs and LSTMs.
It is also worth noting that there are many other ways to generate content than neural networks. For instance, sound and music generation has a long history completely independent of recent AI developments.


Is the error function known or unknown? What is the error function? Is it the same as the cost function?
Is the error function known or unknown?
When I get the outcome of a neural net I compare it with the target value. The difference between both is called the error. When I get mutiple error values e.g. when I pass a batch through the NN I will get as many error value as the size of my batch. Is the error function the plot of the points? If yes, to me the error function would be unknown. I would only know some point on the graph of the error function.

In deep learning, the error function is sometimes known as loss function or cost function (but I do not exclude the possibility that these terms/expressions have also been used to refer to different although related functions, so you should take into account your context).
In statistical learning theory, the function that you want to minimize is known as expected risk
\begin{align}
R(\theta)
&=
\int L(y, f_{\theta}(x)) d P(x, y) \\
&=
\mathbb{E}_{P(x, y)} \left[ L(y, f_{\theta}(x)) \right],
\end{align}
where $\theta$ are the parameters of your model $f$, and the function that you actually minimize is the empirical risk given the dataset $D = \{ (x_i, y_i) \}_{i=1}^n$,
$$
R_{\mathrm{emp}}(\theta)=\frac{1}{n} \sum_{i=1}^{n}L(y_i, f_{\theta}(x_i)),
$$
which is a generalization of the commonly used cost functions, such as mean squared error (MSE) or binary cross-entropy. For example, in the case of MSE, $L = \left(y_{i}-f_\theta(x_i)\right)^{2}$, which can be called the loss function (though this terminology may not be standard).
You optimize the empirical risk, a proxy for the expected risk, because the expected risk is incomputable (given that $P(x, y)$ is generally unknown), so, in this sense, the expected risk is unknown.
If you use the term error function to refer to the expected risk, then, yes, the error function is typically unknown, but error function is typically used to refer to an instance of the empirical risk, so, in that case, it is known and computable.
Note that I purposely used the term loss function above to refer to $L$ and the term cost function to refer to the empirical risks, such as the MSE (i.e., in this case, I did not use loss function and cost function as synonyms), which shows that terminology is not always used consistently, so take into account your context.


Why do we need explainable AI? If the original purpose for developing AI was to help humans in some tasks and that purpose still holds, why should we care about its explainability? For example, in deep learning, as long as the intelligence helps us to the best of their abilities and carefully arrives at its decisions, why would we need to know how its intelligence works?

As argued by Selvaraju et al., there are three stages of AI evolution, in all of which interpretability is helpful.

In the early stages of AI development, when AI is weaker than human performance, transparency can help us build better models. It can give a better understanding of how a model works and helps us answer several key questions. For example why a model works in some cases and doesn't in others, why some examples confuse the model more than others, why these types of models work and the others don't, etc.
When AI is on par with human performance and ML models are starting to be deployed in several industries, it can help build trust for these models. I'll elaborate a bit on this later, because I think that it is the most important reason.
When AI significantly outperforms humans (e.g. AI playing chess or Go), it can help with machine teaching (i.e. learning from the machine on how to improve human performance on that specific task).

Why is trust so important?

First, let me give you a couple of examples of industries where trust is paramount:

In healthcare, imagine a Deep Neural Net performing diagnosis for a specific disease. A classic black box NN would just output a binary "yes" or "no". Even if it could outperform humans in sheer predictability, it would be utterly useless in practice. What if the doctor disagreed with the model's assessment, shouldn't he know why the model made that prediction; maybe it saw something the doctor missed. Furthermore, if it made a misdiagnosis (e.g. a sick person was classified as healthy and didn't get the proper treatment), who would take responsibility: the model's user? the hospital? the company that designed the model? The legal framework surrounding this is a bit blurry.
Another example are self-driving cars. The same questions arise: if a car crashes whose fault is it: the driver's? the car manufacturer's? the company that designed the AI? Legal accountability, is key for the development of this industry.

In fact, this lack of trust, has according to many hindered the adoption of AI in many fields (sources: 1, 2, 3). While there is a running hypothesis that with more transparent, interpretable or explainable systems users will be better equipped to understand and therefore trust the intelligent agents (sources: 1, 2, 3).

In several real world applications you can't just say "it works 94% of the time". You might also need to provide a justification...

Government regulations

Several governments are slowly proceeding to regulate AI and transparency seems to be at the center of all of this.

The first to move in this direction is the EU, which has set several guidelines where they state that AI should be transparent (sources: 1, 2, 3). For instance the GDPR states that if a person's data has been subject to "automated decision-making" or "profiling" systems, then he has a right to access

  "meaningful information about the logic involved"

(Article 15, EU GDPR)

Now this is a bit blurry, but there is clearly the intent of requiring some form of explainability from these systems. The general idea the EU is trying to pass is that "if
 you have an automated decision-making system affecting people's lives then they have a right to know why a certain decision has been made." For example a bank has an AI accepting and declining loan applications, then the applicants have a right to know why their application was rejected.

To sum up...

Explainable AIs are necessary because:

It gives us a better understanding, which helps us improve them.
In some cases we can learn from AI how to make better decisions in some tasks.
It helps users trust AI, which which leads to a wider adoption of AI.
Deployed AIs in the (not to distant) future might be required to be more "transparent".


How do I know if my dataset is ready for a machine learning model? I am new in this area of Machine Learning and Neural Networks. Currently, I'm taking some courses on Udemy and reading a book about it, but I still have one big question regarding data pre-processing.

In all of those Udemy's lessons, people always use a perfect dataset and ready to input in a model. So all you have to do is run it.

How do I know if my dataset is ready for a model? What do I have to do to make it ready? Which evaluations?

I had a few statistics classes in college already and I learned a lot about correlations matrices, autocorrelation functions, and its lags, etc. and I didn't see yet in anywhere someone explaining how can I evaluate my data and then proceed to implement a model to solve my problem.

If anyone could point me a direction, give me some material, show me where I can learn this, anything, it would be really helpful!

Before jumping to modeling, there are a few tasks a data scientist (or ML/AI practitioner) must do:

Ideation (or hypothesizing): Before applying any modeling approach, we need to ask the right questions. We must clearly mention our assumptions and declare how we want to measure the effectiveness of the pipeline. Note that, some tools/algorithms might not fit to the made assumptions or may not lead to the best values in the defined metrics. So, the pipeline must be designed in a way it serves the purpose of answering the defined questions.
Data Cleansing: Real-world data sets are usually not clean. They have all sorts of data issues such as missing value, duplicates, outliers, wrong measurements, fragments, inconsistency, etc. Most of the ML techniques are sensitive (to different extents, of course) to such issues. Therefore, the data should be cleaned before any modeling can be done.
Data Wrangling (or Feature Engineering): In many cases, the gathered data (even cleaned) is not immediately suitable for any modeling/analysis. For example, we may need to convert the documents of a text corpus to vectors of numbers (via TF-IDF or Embedding techniques) before being able to apply a text classifier simply because our classifier only takes numeric data. Converting measurements to other units, breaking addresses to their components, converting times and dates to different formats or timezones are just a few examples of data wrangling tasks (in a broader context, feature engineering may also refer to dimension reduction or feature selection/projection).
Exploratory Data Analysis (EDA): To do the cleansing, understanding the data set features, and ideation, we often need to explore the given data set using visual (e.g., dashboards and diagrams) or summary statistics tools.

Disclaimer: I have no business interest with Udemy. The links are just shared because @pedro-de-sá mentioned they take some courses from Udemy.


Is it possible to classify the subject of a conversation? I would like to classify the subject of a conversation. I could classify each messages of the conversation, but I will loose some imformation because of related messages.
I also need to do it gradually and not at the end of the conversation.
I searched near recurrent neural network and connectionist classification but I'm not sure it answer really well my issue.

This is a difficult problem.
First, how do you define 'subject'? Do you have a (closed) lists of labels you want to assign? What about subjects that overlap, or don't occur in your list? What even is a subject? This is a non-trivial issue.
Second, and this is even harder, how do you want to recognise subjects? A simple solution could be using a list of associated keywords, but this is problematic as many words have multiple meanings, and words are not really a good indicator of a conversation topic in the first place.
Instead of jumping to an implementation method, be clear about how you want to tackle these two items first. Start by annotating a conversation transcript by hand. You will then get a feeling for the problems and possible solutions. After you have done this, you can think about how to get a machine to do it efficiently.
UPDATE: For a scheme to annotate the functions of lines within a conversation have a look at Francis & Hunston (1992) Analysing Everyday Conversation.  In Coulthard, M. (ed.) "Advances in Spoken Discourse Analysis". London: Routledge. pp.123-161.  This is more oriented towards linguistics, but might give you some ideas on how to proceed.


What is a Dynamic Computational Graph? Frameworks like PyTorch and TensorFlow through TensorFlow Fold support Dynamic Computational Graphs and are receiving attention from data scientists.

However, there seems to be a lack of resource to aid in understanding Dynamic Computational Graphs.

The advantage of Dynamic Computational Graphs appears to include the ability to adapt to a varying quantities in input data.  It seems like there may be automatic selection of the number of layers, the number of neurons in each layer, the activation function, and other NN parameters, depending on each input set instance during the training.  Is this an accurate characterization?

What are the advantages of dynamic models over static models?  Is that why DCGs are receiving much attention?  In summary, what are DCGs and what are the pros and cons their use?

Two Short Answers
The short answer from a theoretical perspective is that ...

A Dynamic Computational Graph is a mutable system represented as a directed graph of data flow between operations.  It can be visualized as shapes containing text connected by arrows, whereby the vertices (shapes) represent operations on the data flowing along the edges (arrows).

Note that such a graph defines dependencies in the data flow but not necessarily the temporal order of the application of operations, which can become ambiguous in the retention of state in vertices or cycles in the graph without an additional mechanism to specify temporal precedence.
The short answer from an applications development perspective is that ...

A Dynamic Computational Graph framework is a system of libraries, interfaces, and components that provide a flexible, programmatic, run time interface that facilitates the construction and modification of systems by connecting a finite but perhaps extensible set of operations.

The PyTorch Framework
PyTorch is the integration of the Torch framework with the Python language and data structuring. Torch competes with Theano, TensorFlow, and other dynamic computational system construction frameworks.

———   Additional Approaches to Understanding   ———

Arbitrary Computational Structures of Arbitrary Discrete Tensors
One of the components that can be used to construct a computational system is an element designed to be interconnected to create neural networks.  The availability of these supports the construction of deep learning and back-propagating neural networks.  A wide variety of other systems involving the assembly of components that work with potentially multidimensional data in arbitrarily defined computational structures can also be constructed.
The data can be scalar values, such as floating-point numbers, integers, or strings, or orthogonal aggregations of these, such as vectors, matrices, cubes, or hyper-cubes.  The operations on the generalization of these data forms are discrete tensors and the structures created from the assembly of tensor operations into working systems are data flows.
Points of Reference for Understanding the Dynamic Computation Concept
Dynamic Computational Graphs are not a particularly new concept, even though the term is relatively new.  The interest in DCGs among computer scientists is not as new as the term Data Scientist.  Nonetheless, the question correctly states that there are few well-written resources available (other than code examples) from which one can learn the overall concept surrounding their emergence and use.
One possible point of reference for beginning to understand DCGs is the Command design pattern which is one of the many design patterns popularized by the proponents of object-oriented design.  The Command design pattern considers operations as computation units the details of which are hidden from the command objects that trigger them.  The Command design pattern is often used in conjunction with the Interpreter design pattern.
In the case of DCGs, the Composite and Facade design patterns are also involved to facilitate the definition of plug-and-play discrete tensor operations that can be assembled together in patterns to form systems.
This particular combination of design patterns to form systems is actually a software abstraction that largely resembles the radical idea that led to the emergence of the Von Neumann architecture, central to most computers today.  Von Neumann's contribution to the emergence of the computer is the idea of permitting arbitrary algorithms containing Boolean logic, arithmetic, and branching to be represented and stored as data -- a program.
Another forerunner of DCGs is expression engines.  Expression engines can be as simple as arithmetic engines and as complex as applications such as Mathematica.  A rules engine is a little like DCGs except that rules engines are declarative and meta-rules for rules engines operate on those declarations.
Programs Manipulating Programs
What these have in common with DCGs is that the flow of data and operations to be applied can be defined at run time.  As with DCGs, some of these software libraries and applications have APIs or other mechanisms to permit operations to be applied to functional details.  It is essentially the idea of a program permitting the manipulation of another program.
Another reference point for understanding this principle at a primitive level is the switch-case statement available in some computer languages.  It is a source code structure whereby the programmer essentially expresses, "We're not sure what must be done, but the value of this variable will tell the real-time execution model what to do from a set of possibilities."
The switch-case statement is an abstraction that extends the idea of deferring the decision as to the direction of computation until run time.  It is the software version of what is done inside the control unit of a contemporary CPU and an extension of the concept of deferring some algorithm details.  A table of functors (function pointers) in C or polymorphism in C++, Java, or Python are other primitive examples.
Dynamic Computation takes the abstraction further.  They defer most if not all of the specification of computations and the relationships between them to run time.   This comprehensive generalization broadens the possibilities of functional modification at run time.
Directed Graph Representation of Computation
That's what the Dynamic Computational model is.  Now for the Graph part.
Once one decides to defer the choice of operations to be performed until run time, a structure is required to hold the operations, their dependency relationships, and perhaps mapping parameters.  Such a representation is more than a syntactic tree (such as a tree representing the hierarchy of source code).  Unlike an assembly language program or machine code, it must be easily and arbitrarily mutable.  It must contain more information than a data flow graph and much more than a memory map.  What must that data structure that specifies the computational structure look like?
Fortunately, any arbitrary, finite, bounded algorithm can be represented as a directed graph of dependencies between specified operations.  In such a graph, the vertices (often represented as nodes of various shapes when displayed) represent operations performed on the data and the edges (often represented as arrows when displayed) are digital representations of information originating resulting from some operation (or system input) and upon which other operations (or system output) depend.
Keep in mind that the directed graph is neither an algorithm (in that a precise sequence of operations is specified) nor a declaration (in that data can be explicitly stored and loops, branches, functions, and modules may be definable and nested).
Most of these Dynamic Computational Graph frameworks and libraries permit the components to do computations on the component input that support machine learning.  Vertices in the directed graph can be simulations of neurons for the construction of a neural net or components that support differential calculus.  These frameworks present possibilities of constructs that can be used for deep learning in a more generalized sense.
In the Context of Computer History
Again, nothing mentioned thus far is new to computer science.  LISP permits computational schematics to be modified by other algorithms.  And generalized input dimensionality and numerocity is built into a number of longstanding plug-and-play interfaces and protocols.  The idea of a framework for learning dates back to the same mid-Twentieth Century period too.
What is new and gaining in popularity is a particular combination of integrated features and the associated set of terminology, an aggregation of existing terminology for each of the features, leading to a wider base for comprehension by those already studying for and working in the software industry.

Contemporary (trendy) flavor of API interfaces
Object orientation
Discrete tensor support
The directed graph abstraction
Interoperability with popular languages and packages that support big data, data mining, machine learning, and statistical analysis
Support for arbitrary and systematic neural network construction
The possibility of dynamic neural network structural adaptation (which facilitates experimentation on neural plasticity)

Many of these frameworks support adaptability to changing input dimensionality (number of dimensions and the range of each).
Similarity to Abstract Symbol Trees in Compilers
A dependency graph of inputs and outputs of operations also appears within abstract symbol trees (AST), which some of the more progressive compilers construct during the interpretation of the source code structure.  The AST is then used to generate assembler instructions or machine instructions in the process of linking with libraries and forming an executable.  The AST is a directed graph that represents the structure of data, operations performed, and the control flow specified by the source code.
The data flow is simply the set of dependencies between operations, which must be inherent in the AST for the AST to be used to create execution instructions in assembler or machine code that precisely follows the algorithm specified in the source code.
Dynamic Computational Graph frameworks, unlike switch-case statements or AST models in compilers, can be manipulated in real-time, optimized, tuned (as in the case of plastic artificial nets), inverted, transformed by tensors, decimated, modified to add or remove entropy, mutated according to a set of rules, or otherwise translated into derivative forms.  They can be stored as files or streams and then retrieved from them.
This is a trivial concept for LISP programmers or those that understand the nature of John von Neumann's recommendation to store operational specifications as data.  In this later sense, a program is a data stream to instruct, through a compiler and operating system, a dynamic computational system implemented in VLSI digital circuitry.
Achieving Adaptable Dimensionality and Numerocity
In the question is the comment that one doesn't, "Need to have data set -- that all the instances within it have the same, a fixed number of inputs." That statement does not promote accurate comprehension.  There are clearer ways to say what is true about input adaptability.
The interface between a DCG and other components of an overall system must be defined, but these interfaces may have dynamic dimensionality or numerocity built into them.  It is a matter of abstraction.
For instance, a discrete tensor object type presents a specific software interface, yet a tensor is a dynamic mathematical concept around which a common interface can be used.  A discrete tensor may be a scalar, a vector, a matrix, a cube, or a hypercube, and the range of dependent variables for each dimension may be variable.
It can be the case that the number of nodes in a layer of the system defined in a Dynamic Computational Graph can be a function of the number of inputs of a particular type, and that too can be a computation deferred to run time.
The framework may be programmed to select layer structure (an extension of the switch-case paradigm again) or calculate parameters defining the structure sizes and depth or activation.  However, these sophisticated features are not what qualifies the framework as a Dynamic Computational Graph framework.
What Qualifies a Framework to Support Dynamic Computational Graphs?
To qualify as a Dynamic Computational Graph framework, the framework must merely support the deferring of the determination of algorithm to run time, therefore opening the door to a plethora of operations on the computational dependencies and data flow at run time.  The basics of the operations deferred must include the specification, manipulation, execution, and storage of the directed graphs that represent systems of operations.
If the specification of the algorithm is NOT deferred until run time but is compiled into the executable designed for a specific operating system with only the traditional flexibility provided by low-level languages such as if-then-else, switch-case, polymorphism, arrays of functors, and variable-length strings, it is considered a static algorithm.
If the operations, the dependencies between them, the data flow, the dimensionality of the data within the flow, and the adaptability of the system to the input numerocity and dimensionality are all variable at run time in a way to create a highly adaptive system, then the algorithm is dynamic in these ways.
Again, LISP programs that operate on LISP programs, rules engines with meta-rule capabilities, expression engines, discrete tensor object libraries, and even relatively simple Command design patterns are all dynamic in some sense, deferring some characteristics to run time.  DCGs are flexible and comprehensive in their capabilities to support arbitrary computational constructs in such a way to create a rich environment for deep learning experimentation and systems implementation.
When to Use Dynamic Computational Graphs
The pros and cons of DCGs are entirely problem-specific.  If you investigate the various dynamic programming concepts above and others that may be closely tied to them in the associated literature, it will become obvious whether you need a Dynamic Computational Graph or not.
In general, if you need to represent an arbitrary and changing model of computation to facilitate the implementation of the deep learning system, mathematical manipulation system, adaptive system, or another flexible and complex software construct that maps to the DCG paradigm well, then a proof of concept using a Dynamic Computational Graph framework is a good first step in defining your software architecture for the problem's solution.
Not all learning software uses DCG's, but they are often a good choice when the systematic and possibly continuous manipulation of an arbitrary computational structure is a run time requirement.


Is an embedding a representation of a word or its meaning? What does the term "embedding" actually mean?
An embedding is a vector, but is that vector a representation of a word or its meaning? Literature loosely uses the word for both purposes. Which one is actually correct?
Or is there anything like: A word is its meaning itself?

An embedding is a representation of a word that can be used as a proxy for some of its linguistic properties.
The 'human' representation of a word, a sequence of letters and other symbols, is not related at all to its meaning or use in actual text. It only serves as a look-up key into our cognitive language processing facility (however that actually works) which enables us to understand the meaning in the context of its usage. However, a computer system does not have such a facility.
In order to convert the character string into something more usable for language processing, embeddings are created. These are typically vectors describing other words that surround a word in question (as the meaning of a word depends on its context). There are obvious problems with ambiguities (eg is bank the side of a river or a place where you deposit money?), but there are probably ways around that.
So the embedding (a vector) represents the usage of a word, which strongly correlates with its meaning. Because words as character sequences are useless for most sub-symbolic processing, the terms embedding and word are possibly used interchangably.


How to do machine translation with no labeled data? Can it be possible to train a neural network with no parallel bilingual data?

In this paper: Unsupervised Machine Translation Using Monolingual Corpora Only
 the authors proposed a novel method.

Intuitively it is an autoencoder, but the Start Of Sentence token is set to be the language type.

One other advanced method is to use the pre-training model. In this paper: Cross-lingual Language Model Pretraining researchers proposed an algorithm that utilized the pre-trained multi-lingual trained BERT(with labeled data but we don't need to have a labeled dataset for our task) and the autoencoder mentioned previously.


Is such a captcha AI-resistant? Let's say we have a captcha system that consists of a greyscale picture (of a part of a street or something akin to re-captcha), divided into 9 blocks, with 2 missing pieces.
You need to choose the appropriate missing pieces from over 15 possibilities to complete the picture.
The puzzle pieces have their edges processed with glitch treatment as well as they have additional morphs such as heavy jpeg compression, random affine transform, and blurred edges.
Every challenge picture is unique - pulled from a dataset of over 3 million images.
Is it possible for the neural network to reliably (above 50%) predict the missing pieces? Sometimes these are taken out of context and require human logic to estimate the correct piece.
The chance of selecting two answers in correct order is 1/15*1/14.

Well to give you a short answer, I would say that YES, it would be MORE resistant than a more standard captcha approach...
This being said, I would still go as far as to predict something like a 75-80% successful prediction rates, for a custom model which is designed specifically for defeating a mechanism such as what you describe. The reason why I am fairly confident in such an appraisal, is primarily because of the following:

New techniques which researchers have begun to explore, which are intended to be "Structure Preserving Convolutions" which utilize a higher dimensional filter to store the extra correlation data.

I think that the obfuscation efforts that you mention will definitely help to some degree, although they can be easily defeated by training the model on a dataset which you pull out some portion of the samples during pre-processing and inject the same sort of noise and glitch treatments, etc.

An idea that would be worth exploring would be to process your dataset with an adversarial model which you could then use to generate Adversarial Noise that could then be fed into a pre-process step for your images and replace (or extend) the obfuscation efforts!

TL;DR: If you cant beat 'em, then join 'em! Just train a model to defeat your captcha implementation, and then use the model to generate adversarial examples and then apply obfuscations to your data set accordingly!
For more information on what I am talking about in my suggestion for further obfuscation efforts explore some of the papers you can find on Google Scholar - Ensemble Adversarial Training Examples


What are some possible projects that can be finished in a 4-week time-frame? I am taking AI this semester and we have a semester project that will last 4 weeks. We can choose just about anything.

So, what are some possible semester projects that can be finished in a 4-week time-frame?

Some background information: I am a graduate student in CS, but this is my first AI course. My research area is in the space of data mining and analytics. I am open to doing anything that seems interesting and creative.

Welcome to AI.SE @Kate_Catelena!

I teach AI courses at the undergraduate level, and so have seen a lot of semester projects over the years. Here are some templates that often lead to exciting outcomes:

Pick a new board or card game, and write a program to play it. Your course has probably covered Adversarial Search, and may also have covered Monte Carlo Tree Search, or self-play reinforcement learning approaches. These projects are often fun to mark and creative because they are easy enough to be well done, and yet there are always new, exciting, domains to apply these algorithms to. Some examples of past projects that I thought were neat were an AI to play the boardgame Tac (mostly A* Search), and an AI to play the card game Love Letter (mostly Counter-factual minimax regret, the algorithm used to solve poker).
Pick a question that you would like to know the answer to, that could be addressed with machine learning. Then implement your own ML algorithm (decision tree learners are fairly easy), gather your own data, and show a result. Examples of interesting projects I've seen in the past are using ML to find out which of a number of factors most strongly influenced a students' subjective quality of sleep; and which items are most commonly purchased along with camping supplies (using association rule mining).
Anything involving reinforcement learning. RL projects are always neat to see if accompanied by a visualization showing the learner's behavior at different stages. A strong past project involved a student simply replicating Sutton & Barto's Acrobot experiments with their own implementation of the SARSA-Lambda algorithm.  Other things that might be neat include making a trainable "pet" that the user can influence, or solving games using self-play.
Theoretical results might seem intimidating but are often more accessible than one might think, especially if your discrete math skills are strong. I have had many student projects where the student went away to look at theory papers in ML or Multiagent Systems, found a suggestion in the future work sections that wasn't a big result, but that was fairly easy to prove, and proved it. Sometimes these are even publishable.
Replications. Go find an interesting AI paper (use scholar.google.com), and then see if you can do exactly what the authors suggest and if you get the same result or not. Then, if you have time, see if you can improve on the results. These are often most interesting when you find a paper written in a different field that uses AI. Often the authors of such papers know less about AI than you do, and so it can be fairly easy to improve on their results. I have had several students do projects like this to great effect.

Those categories are a bit vague, but remember: AI touches almost anything. Pick your favorite hobby, and see whether you can relate it to AI using one of the approaches above. Nothing makes a project stand out like one that applies AI to solve some real issues in an exciting domain. Good luck!


speech comment detection by deep speech mozilla for data set I want to create a system so that when a human being says a word or command through a microphone, such as "shut down", the system can execute that command "shut down".
I used the Deep Speech algorithm on the Persian language database, which takes audio data through a microphone and returns the text. The problem I have now is what I have to do from this point on. I need to explain that my system has to work offline and also the number of commands I have is limited.

The problem you state is a well known problem, and it is called "keyword spotting" os KWS. If you add a wake up word before it (like "hey google/siri"), you can also use "voice command" system to alleviate the problem.
There are two kind of KWS systems: those which develop to detect a hard coded set of keywords, and those who are flexible to amend the keyword set. As you state that you trained an acoustic model using deep speech, it looks that you choose the latter approach.
If you trained a deep speech model, you done the easy half of the solution. The other half which is more tricky is to develop a search algorithm based on your acoustic model. I think wfst based algorithms and prefix beam search algorithms are what you should be looking for.
Regards


For each epoch, can I use only on a subset of the full training dataset to train the neural network? If one has a dataset large enough to learn a highly complex function, say learning chess game-play, and the processing time to run mini-batch gradient descent on this entire dataset is too high, can I instead do the following?

Run the algorithm on a chunk of the data for a large number of iterations and then do the same with another chunk and so on?
(Such approach will not produce the same result as mini-batch gradient descent, as I am not including all data in one iteration, but rather learning from some data and then proceeding to learn on more data, beginning with the updated weights may still converge to a reasonably trained network.)

Run the same algorithm (the same model also with only data varying) on different PC's (each PC using a chunk of the data) and then see the performance on a test set and take the final decision as a weighted average of all the different models' outputs with the weight being high for the model which did the best on test sets?

Run the algorithm on a chunk of the data for a large number of iterations and then do the same with another chunk and so on?

  (Such approach will not produce the same result as mini-batch gradient descent, as I am not including all data in one iteration, but rather learning from some data and then proceeding to learn on more data, beginning with the updated weights may still converge to a reasonably trained network.)

This might work if each of the chunks of data individually is still large enough and sufficiently representative of the distribution of the complete population, but probably not the best way to go. In fact, I don't expect it to perform much better than simply using only the very last chunk, and training only on that one. This is because of the following reason. Suppose you first train for a while on chunk A, then for a long time on chunk B, then for a long time on chunk C, etc. While learning on chunk B, there is a significant risk that your model will "forget" everything it learned from chunk A. When learning on chunk C afterwards, it can also "forget" everything learned from chunk B again.

In pseudocode, the approach you proposed here looks as follows:

for each chunk:
    for large number of iterations:
        learn on chunk()

An easy way to improve on that would be to swap the loops around:

for large number of iterations:
    for each chunk:
        learn on chunk()

What I just described there is actually how I interpret "mini-batch gradient descent" though, the chunks would be minibatches (and the minibatches / chunks would be randomly re-selected from the complete population in every iteration of the outer loop, you wouldn't always use the same chunks). Note that this wouldn't be effective if your dataset is so large that it doesn't fit inside your RAM all at the same time, because then you'll have to deal with excessive I/O.

  Run the same algorithm (the same model also with only data varying) on different PC's (each PC using a chunk of the data) and then see the performance on a test set and take the final decision as a weighted average of all the different models' outputs with the weight being high for the model which did the best on test sets?

Yes, this can definitely be effective. This kind of idea (training different models on different subsets of data) is generally referred to as "ensemble" methods. You can even vary the models you use (e.g., have an ensemble with some Random Forests, some SVMs, some Neural Networks, etc.).


What is the difference between human brains and neural networks? There are many people trying to show how neural networks are still very different from humans, but I fail to see in what way human brains are different from neural models in anything but complexity.

The way we learn is similar, the way we process information is similar, the ways we predict outcomes and generate outputs are similar. Give a model enough processing power, enough training samples, and enough time and you can train a human.

So, what is the difference between human (brains) and neural networks?

One incredibly important difference between humans and NNs is that the human brain is the result of billions of years of evolution whereas NNs were partially inspired by looking at the result and thinking "... we could do that" (utmost respect for Hubel and Wiesel).

Human brains (and in fact anything biological really) have an embedded structure to them within the DNA of the animal. DNA has about 4 MB of data and incredibly contains the information of where arms go, where to put sensors and in what density, how to initialize neural structures, the chemical balances that drive neural activation, memory architecture, and learning mechanisms among many many other things. This is phenomenal. Note, the placement of neurons and their connections isn't encoded in dna, rather the rules dictating how these connections form is. This is fundamentally different from simply saying "there are 3 conv layers then 2 fully connected layers...".
There has been some progress at neural evolution that I highly recommend checking out which is promising though.

Another important difference is that during "runtime" (lol), human brains (and other biological neural nets) have a multitude of functions beyond the neurons. Things like Glial cells. There are about 3.7 Glial cells for every neuron in your body. They are a supportive cell in the central nervous system that surround neurons and provide support for and insulation between them and trim dead neurons. This maintenance is continuous update for neural structures and allows resources to be utilized most effectively. With fMRIs, neurologists are only beginning to understand the how these small changes affect brains.

This isn't to say that its impossible to have an artificial NN that can have the same high level capabilities as a human. Its just that there is a lot that is missing from our current models. Its like we are trying to replicate the sun with a campfire but heck, they are both warm.


Is which sense was AlphaGo "just given a rule book"? I was told that AlphaGo (or some related program) was not explicitly taught even the rules of Go -- if it was "just given the rulebook", what does this mean? Literally, a book written in English to read?

it was "just given the rulebook", what does this mean? Literally a book written in English to read?

The program was not given a natural language version of the rules to interpret. That might be an interesting AI challenge in its own way, but none of the current cutting-edge game playing reinforcement learning systems do much in the way of natural language processing.
Instead, "just given the rulebook" is a rough metaphor for what actually happened: The rules of Go were implemented as functions that the game-playing agent could query. The functions can answer things such as "when the board looks like this, what are my valid actions?" and "if the board looks like this, have I won yet?". The board state might be represented by a matrix with stone positions encoded using numbers. Outputs might be a similar matrix of numbers for valid action choices (where the player is allowed to put stones in Go) or perhaps a single number, $0$ for not won yet, $1$ for a move that wins the game.
There may even be further helper functions that help assess moves (e.g. a value for how many enemy stones would be captured if a piece was played in a specific location), but the bare minimum needed is "what moves are valid?" and "has anyone won?". A very common third function, useful for look-ahead planning is "if the board starts like this, and I take that action, what will the board look like next" - with this function, an agent can look ahead to future positions to help search for winning moves.
This approach is common in game playing agents. A learning agent can in theory also learn the rules of the game through trial and error, as long as it receives some feedback when it has broken the rules. However, more often the goal of training the agent is only to play as well as possible - learning the rules from scratch would be extra work for the agent, and maybe not an interesting problem to solve. So the agent is given helper functions by the developers that allow it to explore only valid moves according to the rules.


NLP: What is expected from the output of a perfect coreference system? For instance, consider the following piece of text:
'The father of Richard is a very nice guy. He was born in a poor family. Because of that, Richard learnt very good values. Richard is also a very nice guy. However, Richard's mother embarasses the family. She was born rich and she does not know the real value of the money. She did not have to be a hard worker to succeed in life."
How should a perfect coreference should work? Is this solution the perfect solution?
Cluster 1:
'The father of Richard' (first sentence) <-> 'He' (second sentence)

Cluster 2:
'Richard' (third sentence) <-> 'Richard' (forth sentence)

Cluster 3:
'Richard\'s mother' (fifth sentence) <-> She (sixth sentence) <-> she (sixth sentence) <-> She (seventh sentence)

If I use the coreference library of spacy (neuralcoref), I get these clusters:
Clusters:
[Richard: [The father of Richard, He, Richard, Richard, Richard], a poor family: [a poor family, the family], Richard's mother: [Richard's mother, She, she, She]]

Note that this output says that "Richard" is the same for sentences that is true. However, "He" in the second sentence is not related to "Richard", but to his father. Also "Richard" and the "Father of Richard" are together in the same cluster. Furthermore, "poor family" and "family" should not come together. However, this is realy difficulty since in this case there is some level of ambiguity.'
I know that this is a very difficult problem. The point is not criticize this fantastic library. I am just trying to understand what I should expect as perfect result.
If I change a little the text:
'The mother of Richard is a very nice woman. She was born in a poor family. Because of that, Richard learnt very good values. Richard is a very nice guy. However, Richard's father embarasses the family. He was born rich and he does not know the real value of the money. He did not have to be a hard worker to succeed in life.'
The clusters are:
[Richard: [The mother of Richard, She, Richard, Richard, Richard, He, he, He], a poor family: [a poor family, the family]]

In this case, the clusters become stranger, since "She" and "Richard" are in the same cluster. Furthermore, the "He" related to the "father of Richard" belongs to the cluster, but not "Richard's father".
So, my question is:
What is the perfect result that I should expect from a "perfect" coreference system?

The clusters you mentioned at the beginning seem right. A "perfect" coreference system should be able to find all words/phrases that refer to the same object.
However, in language, there will always be some ambiguity. For example, consider the sentences "Bob and John like building. He has fun doing it.". It is ambiguous whether "He" refers to "Bob" or "John". In that sense, a perfect coreference system may not even exist theoretically. If it did, it would either avoid classifying "He" or recognize that it could go with either "Bob" or "John".


Can intelligent agents have personalities and emotions? Can intelligent agents (and chatbots) have personalities and emotions, given a properly defined ontology?

It can depend on how you define "emotion" and "personality":

Personality

Wikipedia defines personality as "the characteristic set of behaviors, cognitions, and emotional patterns that evolve from biological and environmental factors."  (This is a serviceable definition we can use.)
Since intelligent agents make decisions (take actions), it is safe to say they have a set of behaviors.  Cognition is easy here as well--regardless of the structure of the algorithm, we can say it utilizes a logical process that serves the function of cognition.
Emotion is more difficult.

Emotions

How exactly human emotions work is still partly obscure, and it seems to be a safe bet that chemicals are involved.  But, in a reductionist sense, going back to Ancient Greek Philosophy, we can use a binary "love/hate" (attraction/repulsion) and apply it generally, where these feelings are the prime motivators of natural forces:

And Hesiod says, “First of all things was Chaos made, and then/Broad-bosomed Earth . . . and Love, the foremost of immortal beings,” thus implying that there must be in the world some cause to move things and combine them.
Aristotle (Metaphysics 1.984b)

The real issue comes with interpretations of emotion that involve consciousness, such as on Wikipedia, but even the wiki immediately issues the disclaimer: "Scientific discourse has drifted to other meanings and there is no consensus on a definition," so it may be more useful to look to definitions rooted in cognition and the presumed utility of emotions from an evolutionary standpoint.
If reinforcement learning is being used, an automata can be said to like/dislike (love/hate) certain outcomes, which can be expressed via methods such as reward functions.
Certainly this is not emotion as we humans understand it, but it can be seen as an analog. (Regarding humans chemical response to entities/objects and events/outcomes as reinforce future decisions.)
ANSWER:
In no way do contemporary bots and agents have personalities and emotions in the sense of humans--we are probably still very from this and don't know if it's even possible. It may never be possible validate artificial consciousness, regardless of how conscious a system appears to be.
But, depending on how we define these terms "personality" and "emotion", we can choose to regard automata as having some reduced form of analogous qualities and functions.
Example: If I engineer a game AI to have certain tendencies and behaviors (high-risk-tolerance, partisan, aggressive) that are distinct from other agents (low-risk-tolerance, cooperative, defensive), such agents are commonly referred to as having personalities.
Example: We actually do use terms like "goodness/badness" and informally refer to the automata as "liking" or "not liking" nodes in a graph.  This is a function of anthropomorphization.  (Whether you consider these more than analogies depends on the definitions of feelings, but, generally, there needs to be something to feel. So you also need to reduce consciousness/awareness to the most basic requirement of simply "receiving of input".)


Is a dystopian surveillance state computationally possible? This isn't really a conspiracy theory question. More of an inquire on the global computational power and data storage logistics question.

Most recording instruments such as cameras and microphones are typically voluntary opt in devices, in that, they have to be activated before they start recording. What happens if all of these devices were permanently activated and started recording data to some distributed global data storage?

There are 400 hours of video uploaded to YouTube every minute.

Let’s do some very rough math.

I’m going to assume for the rest of this post that the average video is 1080p which is 2.5GB (or $10^9$ bytes) per hour. From that, we get about 400 hrs * 60 mins * 2.5GB/hrs * 24 hrs = 1.5 petabytes (or $10^{15}$ bytes) per day.

But YouTube videos post are voluntary, and they are far from continuous video streams.

There are about 3.5 billion smartphones in the world. If video was continuously streamed and recorded, going through the same video math above ($3.5 * 10^9 * 1.5 * 10^{15} * 24)$ = 126 yottabytes (or $10^{24}$ bytes) per day.

The IDC projects there will be 175 zettabytes (or $10^{21}$ bytes) in 2025.

Unless my math is very wrong, it would seem as though smartphone cameras alone could produce more data in one day than all of the data created in human history in 2025.

This, so far, has only been about the data recording, but, to implement a surveillance state, all recorded data would need to be processed by AI to intelligent flag data that is significant. How much processing power would be needed to filter 126 yottabytes into relevant information?

Overall, this question is motivated by the spread of dystopian surveillance media like Edward Snowden NSA whistle blowing leaks or George Orwell's sentiment of "Big Brother is Watching You".

Computationally, could we be surveilled, and to what extent? I imagine text messages surveillance would be the easiest, does the world have the computation power to surveil all text messages? How about audio? or video?

You don't necessarily have to analyse it all. Just by having such data available you can achieve a lot in terms of surveillance, as long as you can retrieve relevant parts.

A few years ago there was a Radiolab podcast, "The Eye in the Sky" (there's a full transcript on the site). The basic idea is that you have a plane circling a city 24/7, and filming what goes on. If there was a crime somewhere, you retrieve the recordings after the event, and you can track back to where vehicles involved in the crime were coming from, and where they went after the crime. If nothing happens, you simply archive the data, and perhaps remove it after a month or so.

This method was used to solve a hit-and-run assassination of a police woman who was on her way to work. The gang who committed the attack were rather surprised when the police showed up at their secret hide-out a few days later, as they could see on the images where the cars involved went to later. At the time and place of the murder there were obviously no witnesses who could have done that. And this involved no computational processing at all.

The possibilities this opens up are just scary, as you can track pretty much anybody's movements without actually needing someone to follow them. Add to that street-level CCTV, and not much can happen without you being able to find out.

In this scenario there is no processing at all, but you could imaging simple processing steps, such as tracking vehicles or changes in the environment, which could be used to give clues about potentially 'interesting' events. So instead of using it 'passively' as a kind of memory, you could use that data to identify things that happened that you weren't aware of.

And this is without even any clandestine access to people's data. If you add that dimension, then you might even be able to identify crimes/etc before they even happen. Text processing can be quite fast, but is not easy to do, as presumably few people would openly communicate about things they were planning. So I guess we're still a long way away from that.

Of course there is the ethical dimension (which is mentioned in the podcast): who has access to that data, and who decides what it is used for? If you do, and you suspect your partner of being unfaithful, who/what would stop you from checking out their movements? Or check up on that politician who might have a secret affair, or a gambling issue, or who keeps being in the same locations as a well-known drug dealer. All rather scary.

While a complete analysis of all such data would be very heavy computationally, and fraught with false positives and recall problems, it might simply be enough to index it by time, location, and perhaps people involved (face recognition seems to be reasonably good, though still with a rather high error rate). This is enough already to make me feel worried about the future.


How good is AI at generating new, unseen [visual] examples? By new, unseen examples; I mean like the animals in No Man's Sky.

A couple of images of the animals are:

So, upon playing this game, I was curious about how good is AI at generating visual characters or examples?

We are getting pretty good at image generation, some examples:

Radford, Alec, Luke Metz, and Soumith Chintala. "Unsupervised representation learning with deep convolutional generative adversarial networks." arXiv preprint arXiv:1511.06434 (2015). https://arxiv.org/pdf/1511.06434.pdf
Gregor, Karol, Ivo Danihelka, Alex Graves, Danilo Jimenez Rezende, and Daan Wierstra. "DRAW: A recurrent neural network for image generation." arXiv preprint arXiv:1502.04623 (2015). https://arxiv.org/pdf/1502.04623.pdf

From (1):

Then there is another research direction around  evolutionary algorithms, for example:

Sims, Karl. "Evolving virtual creatures." In Proceedings of the 21st annual conference on Computer graphics and interactive techniques, pp. 15-22. ACM, 1994. https://scholar.google.com/scholar?cluster=6031059536657676358&hl=en&as_sdt=0,22 ; https://www.youtube.com/watch?v=bBt0imn77Zg


Why are Q values updated according to the greedy policy? Apparently, in the Q-learning algorithm, the Q values are not updated according to the "current policy", but according to a "greedy policy". Why is that the case? I think this is related to the fact that Q-learning is off-policy, but I am also not familiar with this concept.

The Q values are updated using a "greedy policy" because, in the Q-learning algorithm, the $\max$ operator is used to determine the "target", which is denoted by
$$\color{green}{R_{t+1}} + \gamma \color{blue}{\max_{a}Q(S_{t+1}, a)}$$
Intuitively, the $\max$ operator is used to take the "greedy" action, that is, the action associated with the highest Q value: $\color{blue}{\max_{a}Q(S_{t+1}, a)}$ means that we are selecting the $Q$ value, associated with the (next) state $S_{t+1}$, which corresponds to the action $a$, such that $Q(S_{t+1}, a)$ is the highest (with respect to other possible actions from $S_{t+1}$).
Note that the $Q$ function receives as input a state and an action. So, for each state $s$, we have an action (among all possible actions, $a_1, a_2, \dots$, which we can take from the state $s$), denote it by $a^*$, such that $Q(s, a^*) > Q(s, a_1)$, $Q(s, a^*) > Q(s, a_2)$, etc. In the expression $\color{blue}{\max_{a}Q(S_{t+1}, a)}$, we are basically selecting $Q(s, a^*)$ for $s = S_{t+1}$.
To explain all the components of this target, I will first have to explain other parts of the Q-learning algorithm. Hopefully, after this explanation, you will be able to understand why Q-learning uses the greedy policy to update the Q values. I will first describe the Q-learning algorithm, you will then, hopefully, understand why it is called a "off-policy" algorithm (which is, IMHO, a quite unintuitive and confusing term to describe what "off-policy" actually means).
Here's the Q-learning algorithm

The Q-learning algorithm estimates (and returns) the $Q$ function associated with a policy $\pi$ (which needs to be passed as input). Intuitively, it does that by simulating an "agent" which takes actions in the environment, "observes" the impact (in terms of rewards received and states visited, after having taken those actions) of those actions on the environment, and then attempts to infer the optimal state-action value (or $Q$) function (that is, the optimal Q function that is associated with the optimal policy, which would give the agent the "highest amount of reward" in the long run), given its interaction with the environment.
Q-learning proceeds in episodes. So, initially, you need to pass the number of episodes as input. You can think of episodes as iterations (like in any iterative optimisation algorithm). However, in the context of RL, an episode is a little bit more specific: the start and end of an episode are associated with specific states of the environment: the episode starts when the agent is in a "starting state" and ends when it is in an "ending (or goal) state". In the pseudocode above, $S_0$ is the starting state.
In the pseudocode above, at the beginning of each episode, we initialise $t=0$, where $t$ represents the time step of a specific episode. Note also that the agent "moves" to the starting state, $S_0$, at the beginning of each episode.
We then have the following loop, which terminates when the agent reaches a goal state (denoted, in the pseudocode, by "terminal", i.e. "terminal state" or "goal state" are synonyms in this context):

So, at each episode, we run the loop above. The block of code inside this loop contains the main logic of the Q-learning algorithm. On each iteration of this inner loop, the agent chooses an action $A_t$ (the action at time step $t$ of the current episode) using any policy (which "ensures that all states are sufficiently visited": you can ignore this for now, given that this answer is already quite long!). In this case, the $\epsilon$-greedy policy is used. How does this $\epsilon$-greedy policy work? If you look at the pseudocode above, $\epsilon$ is initialised at beginning of each episode. In the pseudocode above, $\epsilon$ can change from episode to episode, but assume, for simplicity, that, at every episode, it is a fixed small number (e.g. $0.01$). The statement "Choose action $A_t$ using policy derived from $Q$ (e.g., $\epsilon$-greedy)" means that, with probability $1 - \epsilon$, the "greedy action" is chosen (from the current state, which is, at the beginning of the episode, $S_0$), and, with probability $\epsilon$, a random action is taken. What is the greedy action in this case? It is the action, in the current state, which is associated with the highest Q value (given the current estimate of the Q value). The greedy action is exactly the same action as the action $a^*$ (as I explained above). The difference is that, in this case, we choose $A_t$ using the $\epsilon$-greedy policy: so, most of the times, we choose the greedy action, but, sometimes, we can also choose a random action. Note that the $\epsilon$-greedy policy is indeed a policy, because it allows the agent to choose an action, given a state: this is roughly the definition of a policy.
The agent then executes the just chosen action $A_t$ in the environment, and it observes the "impact" of this action on the environment, which is determined by how the environment responds to this action: the response consists of a reward, $R_{t+1}$, and a next state, $S_{t+1}$.
To recapitulate, the agent chooses an action using the $\epsilon$-greedy policy, executes this action on the environment, and it observes the response (that is, a reward and a next state) of the environment to this action. This is the part of the Q-learning algorithm where the agent interacts with the environment in order to gather some info about it, so as to be able to estimate the Q function.
After that, the agent can update its estimate of the Q function associated with the policy given as input $\pi$. It does that using the following update rule
$$\color{orange}{Q(S_t, A_t)} \leftarrow \color{red}{Q(S_t, A_t)} + \alpha ([\color{green}{R_{t+1}} + \gamma \color{blue}{\max_{a}Q(S_{t+1}, a)}] - \color{red}{Q(S_t, A_t)})$$
where $S_t$ is the current state (of the current episode) the agent is in, $A_t$ is the action chosen using the $\epsilon$-greedy policy (as described above), and $S_{t+1}$ and $R_{t+1}$ are respectively the next state and rewards, which, collectively, are the response of the environment to the just taken action $A_t$.
So, how is the estimate of this $Q$ function updated?
First of all, I would like to note that, if you look at the beginning of the pseudocode above, $Q(s, a)$ is initialized arbitrarily for all states $s \in \mathcal{S}$ and for all actions $a \in \mathcal{A}$: it can e.g. be initialised to $0$. $Q(s, a)$ can e.g. be implemented as a matrix (or 2-dimensional array) $M \in \mathbb{R}^{|\mathcal{S}| \times |\mathcal{A}|}$, where $M[s, a] = Q(s, a)$, $|\mathcal{S}|$ is the number of states in your problem and $|\mathcal{A}|$ the number of actions.
Furthermore, note that the symbol $\leftarrow$ means "assignment" (like assignment to a variable, in the context of programming). So, in the update rule above, we are assigning to $\color{orange}{Q(S_t, A_t)}$ (which will be the next or updated estimate of the Q value for the current state $S_t$ and the just taken action from that state $A_t$) the value $\color{red}{Q(S_t, A_t)} + \alpha (\color{green}{R_{t+1}} + \gamma \color{blue}{\max_{a}Q(S_{t+1}, a)} - \color{red}{Q(S_t, A_t)})$. Let's break this value down.
$\color{red}{Q(S_t, A_t)}$ (on the right side of the assignment) is the estimate of the Q value for the state $S_t$ and action $A_t$ before the assignment. So, we are summing $\color{red}{Q(S_t, A_t)}$ and $\alpha (\color{green}{R_{t+1}} + \gamma \color{blue}{\max_{a}Q(S_{t+1}, a)} - \color{red}{Q(S_t, A_t)})$, and then we assign it to $\color{orange}{Q(S_t, A_t)}$ again.
$\color{green}{R_{t+1}} + \gamma \color{blue}{\max_{a}Q(S_{t+1}, a)}$ is what is often called the "target". Q-learning is a "temporal-difference" (TD) algorithm, and TD algorithms update estimates of the value or action-value functions based on the difference between the current estimate, in the case of Q-learning it is denoted by $\color{red}{Q(S_t, A_t)}$ (on the right side of the $\leftarrow$), and a "target". So, in the Q-learning algorithm, $\color{green}{R_{t+1}} + \gamma \color{blue}{\max_{a}Q(S_{t+1}, a)}$ is the target. We can roughly think of it as "the value that $\color{red}{Q(S_t, A_t)}$ should have been". So, in a certain way, we are performing "supervised learning", where $\color{green}{R_{t+1}} + \gamma \color{blue}{\max_{a}Q(S_{t+1}, a)}$ would be the "ground-truth" label and $\color{red}{Q(S_t, A_t)}$ the current estimate, and so $[\color{green}{R_{t+1}} + \gamma \color{blue}{\max_{a}Q(S_{t+1}, a)}] - \color{red}{Q(S_t, A_t)}$ would be the "error" (or "loss"): in fact, it is often called the "TD error". However, note that this is not really supervised learning, because $\color{green}{R_{t+1}} + \gamma \color{blue}{\max_{a}Q(S_{t+1}, a)}$ is not a ground-truth (it is partially an estimate, because of the part $\gamma \color{blue}{\max_{a}Q(S_{t+1}, a)} $, and it partially a ground-truth, because of $\color{green}{R_{t+1}}$).
To recapitulate, $\color{green}{R_{t+1}} + \gamma \color{blue}{\max_{a}Q(S_{t+1}, a)}$ is the "target", $\color{red}{Q(S_t, A_t)}$ is the current estimate, and $[\color{green}{R_{t+1}} + \gamma \color{blue}{\max_{a}Q(S_{t+1}, a)}] - \color{red}{Q(S_t, A_t)}$ is the "error". We are thus summing the "error" (weighted by the hyper-parameter $\alpha$, which is, in this case, often called the "learning rate") and the current estimate $\color{red}{Q(S_t, A_t)}$ in order to produce the new estimate $\color{orange}{Q(S_t, A_t)}$.
In the target, you can see that we are multiplying the $\color{blue}{\max_{a}Q(S_{t+1}, a)}$ by $\gamma$. This is a hyper-parameter (a parameter which often needs to be chosen by the programmer before the algorithm is executed). It controls the contribution of $\color{blue}{\max_{a}Q(S_{t+1}, a)}$ to the "target": that is, how much of $\color{blue}{\max_{a}Q(S_{t+1}, a)}$ we want to include in the target. Recall that I've just said above that the "target" is composed of the reward $\color{green}{R_{t+1}}$ (which is a "ground-truth" or real-world "experience", because it is directly received from the environment) and $\color{blue}{\max_{a}Q(S_{t+1}, a)}$ (which actually uses an estimate of the Q function, that is, it uses $Q(S_{t+1}, a)$). So, $\gamma$ controls the contribution of an estimate to the "ground-truth".
As I said at the beginning of this answer, $\color{blue}{\max_{a}Q(S_{t+1}, a)}$ can be thought of as the $Q$ value associated with the next state $S_{t+1}$ (which was observed by the agent after he has taken the action $A_t$) and associated with the action $a$, such that $Q(S_{t+1}, a)$ is the highest among all other possible actions from state $S_{t+1}$. In other words, $\color{blue}{\max_{a}Q(S_{t+1}, a)}$ can be thought of the estimate of the Q value associated with the next state $S_{t+1}$ and the "greedy action" taken from that same state.
I would like to note that, at this point, the agent is not really taking actions in the environment. Nonetheless, people, in the RL community, often call Q-learning an "off-policy" algorithm because

It uses the $\epsilon$-greedy policy to interact with the environment (this is often called the "behaviour policy"). In this case, actions are really taken, and the responses of the environment are really produced, observed and used to update estimates of the $Q$ function.

It uses a "target" that is based on an estimate which is "greedy" (i.e. it uses $\color{blue}{\max_{a}Q(S_{t+1}, a)}$).

Given that Q-learning uses estimates of the form $\color{blue}{\max_{a}Q(S_{t+1}, a)}$, Q-learning is often considered to be performing updates to the Q values, as if those Q values were associated with the "greedy policy", that is, the policy that always chooses the action associated with highest Q value. So, you will often hear that Q-learning finds a "target policy" (i.e. the policy that is derived from the last estimate of the Q function) that is "greedy".
I would like to emphasize that, in the Q-learning algorithm, we have several episodes, and, at each episode, the agent interacts with the environment by taking actions and observing the responses of the environment, so as to produce an estimate of the action-value function (i.e. the $Q$ function).


Best Camera and protocol for embedded real time CNN project I'm looking to develop a stand-alone real-time outdoor imaging CNN application, and I can't wrap my head around the myriad of camera options and their communication protocols.
The target is a Linux embedded device (such as beaglebone + edge TPU). and the top language is python. (though if it's unavoidable, I can write some c++ driver)

I have done a number of projects with webcams with OpenCV. However, this project requires a more serious camera, with a motorized zoom and autofocus functions,
with at least 3MP and 30fps.

There are numerous Chinese IP Camera blocks with zoom functions.
but they stream H.265, and I'm not sure how OpenCV would interface with that.
additionally, it's not clear how to control the zoom feature via python in one of them.
Then there are USB2 camera blocks, they seem to be of lower but sufficient quality, but I couldn't find one with motorized zoom.
Then there are Gige and USB3 Vision cameras, they seem optimal but are prohibitively expensive (500$+).

This is kind of an open-ended question, but I couldn't find much information online on this topic, so I hope to find wisdom here.

Thus, I was wondering if someone has advice or recommendations on this.

The Raspberry Pi ZeroW with V2 camera doesn't quite have the specs you need, but it is close: https://picamera.readthedocs.io/en/latest/fov.html#sensor-modes

Here's a waterproof enclosure for it: https://www.youtube.com/watch?v=5CWksss_5lQ


Recent deep learning textbook (i.e. covering at least GANs, LSTM and transformers and attention) I am searching for an academic (i.e. with maths formulae) textbook which covers (at least) the following:

GAN
LSTM and transformers (e.g. seq2seq)
Attention mechanism

The closest match I got is Deep Learning (2016, MIT Press) but it only deals with part of the above subjects.

I recommend Introduction to Deep Learning by Eugene Charniak ISBN 978-0-262-03951-2 (MIT 2018). It mentions GAN & LSTM & Attention (all three occurs in the index).
But read also Pitrat's last book: Artificial Beings: The Conscience of a Conscious Machine - it does cover machine learning (but not in the "deep learning" sense) but was published before 2016.
And see also RefPerSys. If you speak French, also see this and more generally the AFIA organization.


Can a program that requires feedback be considered an AI? If I create a program which takes an input, gives an output and then requires a response to let it know whether the answer it gave was any good does it count as AI?

If not, what is the process of AI? Does it not always need specific parameters? For example, I ask it "Who is the president of the USA?", and I have programmed it to look for news articles in SEOs and remove the "Who" part, is that AI?

There is no "process of AI" as such.  There are many, many different approaches to AI, different ones of which are used in specific applications.

As to whether a purely trial and error approach could be considered AI... I'd offer up a qualified "maybe".  If you do nothing but an exhaustive scan of the solution space, for every trial, then I'd say "No, it's not really any kind of AI".  OTOH, if you're using a knowledge-base of some sort and applying some kind of reasoning (even if it's a heuristic) , and if you have a system that somehow learns from the feedback from the user and gets "smarter" over time, then you're likely working on something that could be considered AI.

All of that said, the exact definition of what is and isn't AI is somewhat fuzzy.   One popular definition is something like "any technology that allows a computer to do something well that currently only humans can do well". So if you're doing something that fits that descriptions, it's possibly an aspect of AI.   And consider again that most people don't really consider "brute force" solutions to be AI.


How do you interpret this learning curve? Loss is MSE; orange is validation loss, blue training loss. The task is NN regression (18 inputs, 2 outputs), one layer 300 hidden units.

Tuning the lr, mom, l2 regularization parameters this is the best validation loss I can obtain. Can be considered overfitting? Is 1 a bad vl loss value for a regression task?

Depends on what does 1 represent in your task.
If you are trying to predict household prices and 1 represents \$1, I think the average validation loss is good. If 1 represents \$10000 in this case, probably something is not right. But remember that there are 2 parts contributing to the overall loss. The mse loss and the l2 penalty loss. (Also remember that most optimizers already implement l2 penalty as weight decay. So you do not need to add it separately)

Some suggestions.

Check if your data has any outliers/ anomalies. Based on your task you should know as to what you can do with these data points. Also see if your dataset has high variance.
If you are worried about over-fitting, think about your data again. Less data + more parameters often leads to over-fitting. If your dataset is too small, you need to think again.
Try to adjust the number of hidden units and observe the results.
Try using cross validation.
Alternatively, try using different optimizers and see what happens (Try Adam).


In the graph search version of A*, can I stop the search the first time I encounter the goal node? I am going through Russel and Norvig's Artificial Intelligence: A Modern Approach (3rd edition). I was reading the part regarding the A* algorithm

  A* graph search version is optimal when heuristic is consistent and tree search version optimal when heuristic is just admissible.

The book gives the following graph search algorithm.

The above algorithm says that pop the node from the frontier set and expand it (assuming not in explored set), and add its children to frontier only if child not in frontier or explored set.

Now, if I apply the same to A* (assuming a consistent heuristic) and suppose I find goal state (as a child of some node) for the first time I add it to the frontier set. Now, according to this algorithm, if the goal state is already in the frontier set, it must never be added again (this implies never be updated/promoted right?).

I have a few questions.

I might as well stop the search when I find goal state for the first time as a child of some node and not wait till I pop the goal state from the frontier?
Does a consistent heuristic guarantee that when I add a node to the frontier set I have found the optimal path to it? (because if I don't update it or re-add it with updated cost (according to the graph search algorithm, the answer to the question must be yes.)

Am I missing something? Because it also states that, whenever A* selects the node for expansion, the optimal path to that node is found and doesn't say that when a node is added to the frontier set, the optimal path is found.

So, I'm pretty confused, but I think the general graph search definition (in the above image) is misleading.

No, the optimal path is found when you pop the goal state. If you stop the search when you first add the goal state then the final path may not be optimal.