Recognize carp and give them a unique id For my internship assignment I have to implement a proof of concept for an application that is supposed to scan a picture with a carp on it and identify which carp this is. All of the carps that are going to be scanned are known and they all exist in the database, so no new carps are scanned.

Is this possible? I've been searching a lot about this topic and the only thing I found is customvision.ai, but for using this I need to have at least 15 pictures of the same carp per tag, but the client only has 1 picture per carp.

What are your recommendations or do you think this is not possible?

I'm not sure I understood your question entirely so please correct me if i'm wrong. You're having all the carp tagged, so if I give you a picture of any of them you know exactly which one is in the picture right? If that is the case then you're dealing with a classic classification problem. One simple way of solving such problem will be to use CNN on the input image to extract features, and at the end of the network have N neurons, where each neuron matches one carp. Just apply a softmax over all the inputs to have a probability distribution, and select the highest value (or if the values is highest then a threshold). Regarding the number of labeled examples per class, having only one might be a problem. I would suggest you to look into few shot learning, which are trying to solve this exact problem (training decent models with limited training data).


Using feature learning for a medical text classification problem I'm currently working with the CHILDES corpus trying to create a classifier that distinguishes children whom suffer from specific language impairment (SLI) from those who are typically developing (TD).

In my readings I noticed that there really isn't a convincing set of features to distinguish the two that have been discovered yet, so I came upon the idea of trying to create a feature learning algorithm that could potentially make better ones.

Is this possible? If so how do you suggest I approach this? From the reading I have done, most feature learning is done on image processing. Another problem is the dataset I have is potentially too small to make it work (in the 100's) unless I find a way to get more transcripts from children.

Having just looked through a few entries from the corpus, I'd personally be skeptical of the applicability of any naive approaches.

In particular light of your small training set, I'd recommend that
whatever method you use should be able to produce human-readable
explanations for the operation of the classifier it builds (e.g.
decision trees/learning classifier systems/genetic programming):
this allows 'common sense' tuning of the classifier, rather than the
danger of overfitting to the training set via black box parameter
optimization.
Rather than throw the entire 'bag of words' at a classifier and hope
that the appropriate set of features will be extracted, you should
first consider what kind of criteria you as a human being might use
to make that decision, and how you might be able to pre-process to
produce features (e.g. syllable-length, metrics from ConceptNet etc) that
are as close to these as reasonably possible.
Having used some human intuition to obtain a reasonable set of
feature primitives, then you can build your classifier and obtain
higher-level expressions that discriminate between them.


Backpropagation equation for a variant on the usual Linear Neuron architecture Recently I encountered a variant on the normal linear neural layer architecture: Instead of $Z = XW + B$, we now have $Z = (X-A)W + B$. So we have a 'pre-bias' $A$ that affects the activation of the last layer, before multiplication by weights. I don't understand the backpropagation equations for $dA$ and $dB$ ($dW$ is as expected).

Here is the original paper that it appeared in (although the paper itself isn't actually that relevant): http://papers.nips.cc/paper/4830-learning-invariant-representations-of-molecules-for-atomization-energy-prediction.pdf

Here is the link to the full code of the neural network: http://www.quantum-machine.org/code/nn-qm7.tar.gz

class Linear(Module):

    def __init__(self,m,n):

        self.tr = m**.5 / n**.5
        self.lr = 1 / m**.5

        self.W = numpy.random.normal(0,1 / m**.5,[m,n]).astype('float32')
        self.A = numpy.zeros([m]).astype('float32')
        self.B = numpy.zeros([n]).astype('float32')

    def forward(self,X):
        self.X = X
        Y = numpy.dot(X-self.A,self.W)+self.B
        return Y

    def backward(self,DY):
        self.DW = numpy.dot((self.X-self.A).T,DY)
        self.DA = -(self.X-self.A).sum(axis=0)
        self.DB = DY.sum(axis=0) + numpy.dot(self.DA,self.W)
        DX = self.tr * numpy.dot(DY,self.W.T)
        return DX

    def update(self,lr):
        self.W -= lr*self.lr*self.DW
        self.B -= lr*self.lr*self.DB
        self.A -= lr*self.lr*self.DA

    def average(self,nn,a):
        self.W = a*nn.W + (1-a)*self.W
        self.B = a*nn.B + (1-a)*self.B
        self.A = a*nn.A + (1-a)*self.A

The forward prop equation is:

$$
Z = (X-A)W - B = XW - AW - B
$$

So the derivatives for $Z$ w.r.t $W$, $A$, $B$ and $X$ should be:

$$
\frac{\partial Z}{\partial W} = X-A  \\
\frac{\partial Z}{\partial A} = - W  \\
\frac{\partial Z}{\partial B} = - 1 \\
\frac{\partial Z}{\partial X} = W
$$

I don't know why he needs the last one though. The first is, like you said, as expected. The other two are wrong, I don't know why he used them in the implementation.


How to understand marginal loglikelihood objective function as loss function (explanation of an article)? I am reading article https://allenai.org/paper-appendix/emnlp2017-wt/ http://ai2-website.s3.amazonaws.com/publications/wikitables.pdf about training neural network and the loss function is mentioned on page 6 chapter 3.4 - this loss function O(theta) is expressed as marginal loglikelihood objective function. I simply does not understand this. The neural network generates logical expression (query) from some question in natural language. The network is trained using question-answer pairs. One could expect that simple sum of correct-1/incorrect=0 result could be good loss function. But there is strange expression that involves P(l|qi, Ti; theta) that is not mentioned in the article. What is meant by this P function? As I understand, then many logical forms l are generated externally for some question qi. But further I can not understand this. The mentioned article largely builds on other article http://www.aclweb.org/anthology/P16-1003 from which it borrows some terms and ideas.

It is said that l is treated as latent variable and P seems to be some kind of probability. Of course, we should assign the greated probability to the right logical form l, but where can I find this assignment. Does training/supervision data should contain this probability function for training/supervision data?

I cannot answer your question but I am stuck in a similar rabbit hole so hopefully these references can help you.
The loss function you are describing would be 0-1 loss. However, 0 would be the if our output matches and 1 would be if it does not. This function is not smooth and not convex. Thus we often replace it with a surrogate loss function such as log likelihood.
You can read more about surrogate loss function on pg 269 of Deep Learning by Ian Goodfellow available here: https://www.deeplearningbook.org/
or here : https://en.wikipedia.org/wiki/Loss_functions_for_classification
I did not have time to read the article, but the reason that you are seeing a probability is that they are using a Bayesian framework for neural networks. This was first described in A Practical Bayesian Framework for Backpropagation networks by McKay 1992.
It is explained very well in a video series by Hinton on youtube here:
https://www.youtube.com/watch?v=YcwZFNd3UvI
Hopefully these references help you I wish I could explain in greater detail.


Would this neural network have short term memory? I want to design a NN that can remember it's last 7 actions and use them as inputs. So for example it would be able to store words in it's memory. Therefore if it had a choice of 10 different actions, the number of words it could store is $10^7$.

Here is my design:

$$out_{n+1} = f(out_n, in_n)\mathbf{N} + out_n.\mathbf{M}$$

$$action_n = \sigma(\mathbf{N} \cdot out_n)$$

Where $f$ represents some layered neural network. Some of the actions would be physical actions and some might be internal (such as thinking of the letter 'C').

Basically I want $out_n$ to be an array that keeps the last 6 action values and puts them back in. So $M$ will be the matrix:

$$\begin{bmatrix}
0&1&0&0&0&0\\
0&0&1&0&0&0\\
0&0&0&1&0&0\\
0&0&0&0&1&0\\
0&0&0&0&0&1\\
0&0&0&0&0&0
\end{bmatrix}$$

i.e. it would drop the 6th item from it's memory.

and $N$ would be the vector:

$$\begin{bmatrix}
1&0&0&0&0&0&0
\end{bmatrix}$$

I think this would be equivalent to an equation of the form:

$$out_{n+1}=F(in_n,out_n,out_{n-1},out_{n-2},...,out_{n-6})$$

So I think this would be an advantage over an RNN since this model remembers precisely it's last 6 actions. But would this be better than an RNN or worse? One could increase it's memory to more than 7 quite easily.

I think it's basically the same archececture as an RNN except elinimating a lot of the connections. Is this a new design or a common design?

One problem with this design is that you might also want a memory that is over longer time periods (e.g. for actions that take more than one tick.) But that might be solved by enhancing the archecture.

Congrats, you have invented 1d convolution. Convolution combined with RNN would have some advantage over just RNN. Think about the perception field.
In this layer, you do aggregate $6$ values to one. Imagine two of them - it will be $36$ already, etc. But, in the end, you still need RNN at the end to aggregate a variable length to constant length.


How to forecast multiple target attributes in Python? I need to forecast two non-correlated time-series (non-stationary). A sample is presented below:
414049364,21773560
414049656,21773926
414049938,21774287
414050204,21774638
414050453,21774975
414050682,21775296
414050895,21775597
414051093,21775874
414051278,21776125
414051453,21776344
414051620,21776530
414051780,21776678
414051935,21776785
414052089,21776849
414052242,21776865

The above is the input (two attributes) and the output (prediction) is composed of two targets (the same as input) for instance,
414052252,21776765

However, current regression techniques only consider a single attribute (class) forecasting but two or more. I've checked the following site https://machinelearningmastery.com/multi-output-regression-models-with-python/ for multi-target regression or predictive clustering trees. Unfortunately, I don't know how to adapt my data to those techniques. Ideally, I would like to predict multiple steps.
Any idea?

In general, multi output models is not that different.
I.e.

As Raghu mentioned in commentary, you could train separate model for each output. There is even helper module in sklearn for that (MultiOutputRegressor)
DecisionTreeRegressor from sklearn allows multiple outputs out-of-the box
Any neural network framework allows any number of outputs

In your particular case there are two much bigger factors.

It's timeseries. Timeseries requires whole another set of models then other data, i.e ANOVA, RNN or transformers.
You data is not well scaled. The reason is that the change between points is insignificant in comparison with their absolute values. You need to deal with it -- but exact solution would depend on your particular task. You could try to substract the mean value on train set or predict the difference, not absolute value.

Edit: here is simple example
import numpy as np
raw = np.array([
[414049364,21773560],
[414049656,21773926],
[414049938,21774287],
[414050204,21774638],
[414050453,21774975],
[414050682,21775296],
[414050895,21775597],
[414051093,21775874],
[414051278,21776125],
[414051453,21776344],
[414051620,21776530],
[414051780,21776678],
[414051935,21776785],
[414052089,21776849],
[414052242,21776865]
])

X = raw[:-1]
y = raw[1:]
from sklearn.linear_model import LinearRegression
model = LinearRegression()
# Train the model
model.fit(X, y)
# Try to predict next step
model.predict([[414051780,21776678]])
# The predicted value [414051933,  21776806] and the true value would be [414051935,21776785], not so far

Next you could normalize the values, use few previous steps as a features, use more complex algorithm then linear regression, and make train/eval split for better model evaluation.


Is it possible to separately evolve a part of the population? In a classic example of a genetic algorithm, you would have a population and a certain amount of simulation time to evaluate it and breeding. Then proceed to the next generation.

Is it possible, during the simulation process, to have an isolated and small part of the population and keep it evolving in their own little island for some time while the rest of the population continues to evolve normally?
After that, they could be reunited with the rest of the population and the end of the simulation would go through. After that, breed the population and continue.

This is a super important part of natural evolution and probably some know if it actually works with genetic programming?

There have been extensive studies within evolutionary computation in the area of island models and niching for doing exactly this.
The advantages of this approach include greater population diversity (which is particularly useful when the problem is multiobjective) and the potential for concurrent execution of each separate population.
See also the answers to the question What is the niching scheme?.
With specific reference to genetic programming, here is a recent paper that uses a parallel island model.


Chatbot using Machine Learning vs Programming I am currently building a chatbot. What I have done so far is, collected possible questions/training data/files and created a model out of it using Apache OpenNLP; the model is able to predict all the questions that are in the training data and fails to predict for new questions.

Instead of doing all the above, I can write a program that matches the question/words against training data and predict the answer — what is the advantage of using Machine Learning algorithms?

I have searched extensively about this and all I got was, in Machine Learning there is no need to change the algorithm and the only change would be in the training data, but that is the case with programming too: the change will be in training data.

In my view ML does not work very well for conversational AI systems. It is generally alright for intent recognition, so getting what the user wants if they ask a question ("I want to book a flight?", "What is the weather in London?"), but anything after that quickly becomes difficult to handle, especially multi-step conversations that go beyond simple question/answer pairs.

My suggestion would be to plan possible dialogues out as flow charts (more like trees/graphs, as there can be multiple branches at any point), and then write a program that interprets the graph based on user input and gives appropriate replies. You will also want to have some conversational memory to keep track of any information the user has mentioned. That is also tricky to do in a ML system.

For a very simple framework to start off with, have a look at ELIZA. It's half a century old, but you can still use it as a starting point.

(Disclaimer: I work for a company that makes conversational AI systems)


How will an AI comprehend the ethics of "right" and "wrong"? Here is one of the most serious questions, about the artificial intelligence.
How will the machine know the difference between right and wrong, what is good and bad, what is respect, dignity, faith and empathy.
 A machine can recognize what is correct and incorrect, what is right and what is wrong, depend on how it is originally designed.
It will follow the ethics of its creator, the man who originally designed it
 But how to teach a computer something we don't have the right answer.
 People are selfish, jealous, self confident. We are not able to understand each other sorrows, pains beliefs. We don't understand different religions, different traditions or beliefs.
Creating an AI might be breakthrough for one nation, or one race, or one ethnic or religious group, but it can be against others.

Who will learn the machine a humanity?   :)

Right and wrong only exist relative to some goal or purpose.

To make a machine do more right than wrong, relative to human goals, one should minimize the surface area of the machine's purpose. Doing that minimizes the intrinsic behavior of the AI, which enables us to reason about the right and wrong behaviors of the AI, relative to human purposes.

Horses are quite general over the domains of their purposes, but are still predictable enough for humans to control and benefit from. As such, we will be able to produce machines (conscious or unconscious) that are highly general over particular domains, while still being predictable enough to be useful to humans.

The most efficient machines for most tasks, though, will not need consciousness, nor even the needs that cause survivalistic, adversarial and self-preserving behaviors in eukaryotic cells. Because most of our solutions won't need those purposes to optimize over our problems, we can allow them to be much more predictable.

We will be able to create predictable, efficient AIs over large domains that are able to produce predictable, efficient AIs over more specific domains. We'll be able to reason about the behavioral guarantees and failure modes of those narrow domains.

In the event that we one day desire to build something as unpredictable as a human, like we do when having babies, then we'll probably do that with the similar intentions and care that we use to bring an actual baby into the world. There is simply no purpose in creating a thing more unpredictable than you unless you're gambling on this thing succeeding you in capability - which sounds exactly like having babies.

After that, the best we can do is give it our theories about why we think we should act one way or another.

Now, theoretically, some extremely powerful AI could potentially construct a human-like simulacrum that, in many common situations, seems to act like a human, but that in fact has had all of it's behaviors formally specified a priori, via some developmental simulation, such that we know for a fact that all such behaviors produce no intentional malice or harm. However, if we can formally specify all such behaviors, we wouldn't be using this thing to solve any novel problems, like curing cancer, as the formal specification for curing cancer would already have been pre-computed. If you can formally specify the behaviors of a thing that can discover something new, you can just compute the solution via the specification, without instantiating the behaviors at all!

Once AI has reached a certain level of capability, it won't need to generate consciousnesses to derive optimal solutions. And at that point, the only purpose for an artificial human to exist will be, like us, for its own sake.


What is the difference between ConvLSTM and CNN LSTM? What will be the difference when used for video classification? Will they yield different results or are they the same fundamentally?

After doing a bit of research I found that the LSTM whose gates perform convolutions is called ConvLSTM.

The term CNN LSTM is loose and may mean stacking up LSTM on top of CNN for tasks like video classification

Reddit thread discussing this


Could machine learning be used to measure the distance between two objects from a picture or live camera? Could machine learning be used to measure the distance between two objects from a picture or live camera?

An example of this is the measurement between the centre of each eye pupil.

This area is all new to me, so any advice and suggestions would be greatly appreciated.

The short answer is: yes, it could. In what you are describing, there's nothing very new or specific conceptually; it sounds like a standard regression task. Now the problem that you're actually facing is: do you have the data?

Algorithms won't be able to learn the distance between eyes if you don't have the data that it takes. It could be supervised labels (1 distance per image which would be your regression target), reconstruction from depth maps, multi-view estimation etc. There's a number of ways you could do that given the appropriate data.

People focus on algorithms a lot, and that's good. But taking a good look at your data is often as important (if not more).

Now a good example would be in the self-driving car literature. You could start with this blog-post and go through the papers they reference: https://towardsdatascience.com/vehicle-detection-and-distance-estimation-7acde48256e1

There also seems to be some litterature about your eyes examples (https://arxiv.org/pdf/1806.10890.pdf, https://www.sciencedirect.com/science/article/pii/S0165027019301578) so skimming through these papers & the datasets they use could guide to towards answering my question: is there data for this task?


String node untying algorithm Let say that we have four straight string with colors RED, GREEN, BLUE and YELLOW. These strings are tied up randomly. We know the current state of string (like starting point, where we should we start untying from) and the final state (where current string must look like after untying them).

Here is a very simple example of the problem:

At every move, we are allowed to replace only two nearby strings. For example, to solve the shown problem, we should do the following replacements:

1) Replace GREEN and YELLOW

2) Replace BLUE and GREEN

But in a programming environment, how can I calculate the movements to untie the strings? What will be the algorithm specifically to solve such type of problems? Here is another variation of the question. Its not possible to solve this problem according to the answer.

Sequential programming would not be suitable for this kind of problem, but an algorithm could be implemented in a declarative programming language. I would suggest using Answer Set Programming, a language that is designed for logic axioms.


Applying Artificial neural network into kaggle's house prices data set gave bad predicted values I am trying to solve the kaggle's house prices using neural network. I've already made it with ensembling several models (XGBoost, GradientBooster and Ridge) and I've got a great score ranking me between the top 25%.

I imagined that by adding a new model to the ensembled models like ANN would increase prediction accuracy, so I did the following:

import keras

model = keras.models.Sequential()

model.add(keras.layers.Dense(235, activation='relu', input_shape=(235,)))
model.add(keras.layers.Dense(235, activation='relu'))
model.add(keras.layers.Dense(235, activation='relu'))
model.add(keras.layers.Dense(235, activation='relu'))
model.add(keras.layers.Dense(235, activation='relu'))
model.add(keras.layers.Dense(1))

model.compile(optimizer='adam', loss='mean_squared_error')
model.summary()
model.fit(dataset_ann, y, epochs=100, callbacks=[keras.callbacks.EarlyStopping(patience=3)])
y_pred = model.predict(X_test_ann)

I choosed 235 neurons for each layer, as the training set has 235 features.

For model ensembling:

y_p = (0.1*model.predict(X_test_ann)+0.2*gbr.predict(testset)+0.3*xgb.predict(testset)+0.1*regressor.predict(testset)+0.1*elastic.predict(testset)+0.2*ridge.predict(testset))

The shape of y_p is (1459, 1459) instead of (1459, ) where columns are all having the same values, so taking y_p[0] would be more than enough.

I submitted the result into kaggle and went from top 25% into bottom 60%.

Is it because the number of hidden layers with its input? Or because there is few data to train (1460 rows of train set) and the neural network needs more than that? Or is it because of the number of neurons in each layer?

I tried with epoch = 30, 100, 1000 and got nearly the same bad ranking.

Is it because the number of hidden layers with its input? Or because there is few data to train (1460 rows of train set) and the neural network needs more than that? Or is it because of the number of neurons in each layer?

I think you are onto something here. You have over a thousand nodes, almost as many as training samples you have. In my experience, the network will have no trouble overfitting, given you train long enough. If you dont train long enough on the other hand, your network will probably not learn anything.

I would try with a smaller network, so you "force" it to learn something, instead of memorizing.


How to speed up YOLOv3 detection speed? I want to implement YOLO V3. I want to know which framework will give me a faster result.
What are the advantages of implementing YOLO V3 on the darknet framework vs Keras framework?

Darknet is "native" framework, so basically, you don't need to implement anything, all code for yolov3 is available at their github repo, you just need to figure it out, play with it. Keras, in my opinion, is not flexible enough to easily implement yolo. If you want to implement yolo from scratch I would probably go with PyTorch it has a dynamic graph + more integration with python. Regarding speed, it's hard to say objectively which framework is faster, especially comparing with darknet, since probably nobody did a comparative analysis. The easiest solution to speed up yolo3 is obviously buying more powerful GPU :D.

Also, in my understanding what they did in yolov3 is that they intentionally sacrificed speed in order to be able to detect smaller objects, so if you don't care too much about small grouped up objects go with yolov2 it is very fast and has a pretty decent mAP.


Can multiple activation functions be replaced with a single activation function? I'm just started to learn deep learning and I have a question about this neural network:

I think $h_1$, $h_j$ and $h_n$ are perceptrons. So, if they are perceptrons, all of them will have an activation function.

I'm wondering if it is possible to have only one activation function, and sum the output to all of the perceptrons, and pass that sum to that activation function. And the output of this activation function will be $y$.

I will have this network, where $H1$, $Hj$ and $Hn$ don't have activation function:

The input for the activation function will be the sum of the outputs of $H1$, $Hj$ and $Hn$ without been processed by an activation function.

Is that possible (or is it a good idea)?

TL;DR: This is possible but removing the activations will decrease the expressivity of the network because it will become mathematically equivalent to a single neuron.

Mathematical Explanation

The outputs of your intermediate neurons (in the absence of activation functions) are now:

$\text{(1)}\quad H_i(x) = \sum_{j=1}^nw_{ij}\cdot x_j+b_i$

You are then summing each $H_i$:

$\text{(2)}\quad\mathcal{H(x)}=\sum_{i=1}^nH_i(x)$

You then pass $\mathcal{H(x)}$ to some activation say $g$:

$\text{(3)}\quad\hat y = g(\mathcal{H(x)})$

The trouble is that the inner term $\mathcal{H(x)}$ mathematically reduces to a single linear operation on $x$. Proof:

$\mathcal{H(x)}=\sum_{i=1}^nH_i(x)$. Substituting in (1):
$\mathcal{H(x)}=\sum_{i=1}^n((\sum_{j=1}^nw_{ij}\cdot x_j)+b_i)$. This can be re-aranged :
$\mathcal{H(x)}=(\sum_{j=1}^n(\sum_{i=1}^nw_{ij})\cdot x_j)+\sum_{i=1}^nb_i$. But this reduces to:
$\mathcal{H(x)}=\sum_{j=1}^n\tilde w_{j}\cdot x_j+\tilde b$. Where $\tilde w_{j},\tilde b$ are scalars.

Thus, without the non-linear activations (3) mathematically reduces to a single neuron.


Does the model learn from the average of all the data points in the mini-batch? I used the example at - https://github.com/aymericdamien/TensorFlow-Examples/blob/master/examples/5_DataManagement/tensorflow_dataset_api.py - to create my own classification model. I used different data but the basic outline of datasets was used.

It was important for my data type to shuffle the data and then create the training and testing sets. The problem, however, comes as a result of the shuffling.

When I train my model with the shuffled train set I get a +- 80% accuracy for train and +- 70% accuracy for the test set. I then want to input all the data (i.e. the set that made the training and test set) into the model to view the fully predicted output of this data set that I have.

If this data set is shuffled as the training and testing set was I get an accuracy of around 77% which is as expected, but then, if I input the unshuffled data (as I required to view the predictions), I get a 45% accuracy. How is this possible?

I assume it's due to the fact that the model is learning incorrectly and that it learns that the order of the data points plays a role in the prediction of those data points. But this shouldn't be happening as I am simply trying to (like the MNIST example) predict each data point separately. This could be a mini-batch training problem.

In the example mentioned above, using data sets and batches to train, does the model learn from the average of all the data points in the mini-batch or does it think one mini-batch is one data point and learn in that manner (which would mean order matters of the data)?

Or if there are any other suggestions.

if I input the unshuffled data (as I required to view the predictions) I get a 45% accuracy. How is this possible?

When you build your dataset and then split it into training, test, and validation sets, you have to be sure that the training encompasses aspects of the test set or the validation set and more. That's why shuffling the data is important.
In the case where you didn't shuffle your dataset, your NN was confronted to object it never sees before (i.e. in the training set) and then couldn't classify it properly.

  does the model learn from the average of all the data points in the mini-batch or does it think one mini-batch is one data point and learn in that manner (which would mean order matters of the data)?

When using a mini batch you decide the size (b_size) of the mini batch and the number of mini batches (n_batch). Then, for n_batch times you will draw b_size random indices from X_train and y_train arrays, build with these indices X_batch and y_batch, and tune your model parameters on the 2 latter arrays (and do it n_batch times).

So having shuffled your dataset and takes randomly parts of your dataset to train your model ensure you to get rid of correlations between two points of your datasets and better generalization


What is an artificial neural network? What is an artificial neural network in artificial intelligence?

It is apparently used to find patterns in data and it is loosely inspired by human neural networks.

An (artificial) neural network (NN) can be described at different levels of detail, but, essentially, an NN is a data structure (or model), which behaves like a function: that is, it can receive one or more inputs and it produces one or more outputs.

Most of the NNs can be thought of as graph-like data structures that contain nodes which are connected by weighted edges, where some of the nodes are associated with the inputs and some of the nodes are associated with the outputs. The remaining nodes are often called "hidden nodes". However, there are NNs which might not be easily understood as a graph-like structure: for example, convolutional neural networks.

The number of nodes and the connections between them defines the architecture of the NN and, intuitively, its "expressivity", that is, the number or type of functions it can approximate. Each node in an NN often computes a "function" of its inputs: for example, it could sum the inputs.

The weights of the edges connecting the nodes can change, during a "learning or training phase", so that the NN produces the desired outputs given certain inputs. For example, we can train an NN to "approximate" the function $f(x) = x$. Thus, an NN is often described as a "function  approximator".

There are several algorithms that can be used to "train" an NN to approximate a desired (and often unknown) function. The most common one is based on an optimisation algorithm (like gradient descent) and the back-propagation algorithm. In order to approximate a function, an NN requires data, often in the form of pairs of inputs and outputs, $(x, y)$, where $x$ is the input and $y$ is the output of the function $f$ (that we want to approximate). Intuitively, after the learning phase, an NN that receives an input similar to $x$, denoted by $x'$, will then produce an output similar to $y$, $y'$.

There are a lot of tasks that can be thought of as functions. For example, the task of machine translation can be thought of as a function $f(x) = y$, where the input, $x$, is a sentence in one language (e.g. English) and the output, $y$, is a sentence in another language (e.g. Chinese). In this case, $f$ is unknown (to a lot of people), but an NN can still learn (from the data) an approximation of it. Therefore, NNs have wide applicability and are thus quite useful.


SEIF motion update algorithm doubt I want to implement Sparse Extended information slam. There is four step to implement it. The algorithm is available in Probabilistic Robotics Book at page 310, Table 12.3.

In this algorithm line no:13 is not very clear to me. I have 15 landmarks. So $\mu_t$ will be a vector of (48*1) dimension where (3*1) for pose. Now $H_t^i$ is a matrix whose columns are dynamic as per the algorithm it is (3j-3) and 3j. J is the values of landmarks 1 to 15. Now how could I multiply a dynamic quantity with a  static one. There must be a error that matrix dimension mismatch when implement in matlab.

Please help me to understand the algorithm better.

You are right, that pseudocode is not correct. In particular, the definition of $H_t^i$ in line $11$ should be changed; all the way on the right-hand side, it should have $3N - 3j$ columns of $0$s, rather than $3j$ columns of $0$s.

With that change, every matrix $H_t^i$ will have the same number of columns:

$$6 + 3j - 3 + 3N - 3j = 3 + 3N,$$

which evaluates to a total of $48$ in your case (because you have $N = 15$ landmarks). That's precisely the correct dimensionality required for matrix multiplication with your $\mu_t$ vector.

The version of the book that you linked to appears to be a fairly old draft. This webpage contains errata for the third edition for the book, in which page 393 corresponds to what was page 310 in your version of the book. The errata for that third edition of the book can be downloaded at the following URL: http://probabilistic-robotics.informatik.uni-freiburg.de/corrections/pg393.pdf

There you'll find the fix that I described above, but also some other fixes (most of them are just notational, adding bars over the $\mu$ vectors, but it looks like a more serious issue was additionally fixed in line 13, where a minus was changed to a plus).


Is there any use of using 3D convolutions for traditional images (like cifar10, imagenet)? I am curious if there is any advantage of using 3D convolutions on Images like Cifar10/100 or Imagenet. I know that they are not usually used on this data set, though they could because the channel could be used as the "depth" channel.

I know that there are only 3 channels, but lets think more deeply. They could be used deeper in the architecture despite the input image only using 3 channels. So we could have at any point in the depth of the network something like $(C_F,H,W)$ where $C_F$ is dictated by the number of filters and then apply a 3D convolution with kernal size less than $C_F$ in the depth dimension.

Is there any point in doing that? When is this helpful? When is it not helpful?

I am assuming (though I have no mathematical proof or any empirical evidence) that if the first layer aggregates all input pixels/activations and disregards locality (like a fully connected layer or conv2D that just aggreagates all the depth numbers in the feature space), then 3D convs wouldn't do much cuz earlier layers destroyed the locality structure in that dimension anyway. It sounds plausible but lack any evidence or theory to support it. I know Deep Learning uses empirical evidence to support its claims so perhaps there is something that confirms my intuition?

Any ideas?

Similar posts:

https://forums.fast.ai/t/are-there-any-successful-vision-models-that-use-3d-convolutions/52503/2
When should I use 3D convolution?
3D convolution Neural Nets

As far as I've seen there is no use of 3D CNNs for traditional image classification tasks.

The reason I think is that, while these images do have multiple channels, there is no spatial information in those channels for the 3D convolution to extract. On the other hand it makes more sense to take the weighted sum of those pixels along that dimension (as the 2D convolution does).

3D CNNs have been used, as far as I know, only for applications where you have volumetric data, i.e. the images are sequential and, when combined, form a large 3D image.


Given two neural networks that compute two functions $f(x)$ and $g(x)$, how can I create a neural network that computes $f(x)g(x)$? I have two functions $f(x)$ and $g(x)$, and each of them can be computed with a neural network $\phi_f$ and $\phi_g$.
My question is, how can I write a neural net for $f(x)g(x)$?
So, for example, if $g(x)$ is constant and equal to $c$  and $\phi_f = ((A_1,b_1),...(A_L,b_L))$, then $\phi_{fg} = ((A_1,b_1),...,(cA_L,cb_L))$.
Actually, I need to show it for $f(x)=x$ and $g(x)=x^2$ if this make something easier.

Use another network h which takes f(x) and g(x) as input i.e. h(f(x), g(x)).
Training psuedo code(pytorch):
for epoch in epochs:
    for batch, x in dataset:
        train(f)
        train(g)

for epoch in epochs:
    for batch, x in dataset:
        # freeze f and g (using torch.no_grad in pytorch)
        with torch.no_grad:
            fx = f(x)
            gx = g(x)
        pred = h(fx, gx)
        loss = loss_fn(pred, (fx . gx))
        backpropagate()
        optimise()


Is it ok to struggle with mathematics while learning AI as a beginner? I have a decent background in Mathematics and Computer Science .I started learning AI from Andrew Ng's course from one month back. I understand logic and intuition behind everything taught but if someone asks me to write or derive mathematical formulas related to back propagation I will fail to do so.
I need to complete object recognition project within 4 months.
Am I on right path?

I think the key part of your question is "as a beginner". For all intents and purposes you can create a state of the art (SoTA) model in various fields with no knowledge of the mathematics what so ever.

This means you do not need to understand back-propagation, gradient descent, or even mathematically how each layer works. Respectively you could just know there exists an optimizer and that different layers generally do different things (convolutions are good at picking up local dependencies, fully connected layers are good at picking up connections among your neurons in an expensive manner when you hold no previous assumptions), etc.. Follow some common intuitions and architectures built upon in the field and your ability to model will follow (thanks to the amazing work on opensource ML frameworks -- Looking at you Google and Facebook)! But this is only a stop-gap.

A Newton quote that I'm about to butcher: "If I have seen further it's because I'm standing on the shoulders of giants". In other words he saw further because he didn't just use what people before him did, he utilized it to expand even further. So yes, I think you can finish your object detection project on time with little understanding of the math (look at the Google object detection API, it does wonders and you don't even need to know anything about ML to use it, you just need to have data. But, and this is a big but, if you ever want to extend into a realm that isn't particularly touched upon or push the envelope in any meaningful way, you will probably have to learn the math, learn the basics, learn the foundations.


How are weights for weighted x-entropy loss on imbalanced data calculated? I am trying to build a classifier which should be trained with the cross entropy loss. The training data is highly class-imbalanced. To tackle this, I've gone through the advice of the tensorflow docs

and now I am using a weighted cross entropy loss where the weights are calculated as

weight_for_class_a = (1 / samples_for_class_a) * total_number_of_samples/number_of_classes

following the mentioned tutorial.

It works perfectly, but why is there this factor total_number_of_samples/number_of_classes?
The mentioned tutorial says this

  [...] helps keep the loss to a similar magnitude.

But I don not understand why. Can someone clarify?

This comes from the fact that you want the same magnitude from the loss. Think of it this way: a non-weighted loss function actually has all its weights to 1 and so over the whole data set, samples are weighted with 1 and the sum of all weights is therefore $N$, if $N$ is the total number of samples.

Now in the case of a weighted loss, we want the weights to also sum to $N$ so that the loss's magnitude is comparable ($i = 1..C$ are your classes, $N_i$ is the number of samples for class $i$):

$$S = \sum_{i=0\ }^{C} \sum_{s_i=1}^{N_i} w_{i} = \sum_{i=0}^{i}\sum_{s_i=1}^{N_i}\frac{1}{N_i} \frac{N}{C} = \frac{N}{C} \sum_{i=0}^{C}\sum_{s_i=1}^{N_i}\frac{1}{N_i} = \frac{N}{C} \sum_{i=0}^{C}N_i\frac{1}{N_i}  = \frac{N}{C} \sum_{i=0}^{C}1 = \frac{N}{C} C = N$$


Can neuro-evolution of augmenting topologies (NEAT) neural networks be built in TensorFlow? I am making a machine learning program for time series data analysis and using NEAT could help the work. I started to learn TensorFlow not long ago but it seems that the computational graphs in TensorFlow are usually fixed. Is there tools in TensorFlow to help build a dynamically evolving neural network? Or something like PyTorch would be a better alternative? Thanks.

Check this implementation in TensorFlow Eager: https://github.com/crisbodnar/TensorFlow-NEAT


Neural network Model to infer inputs given an output I have a data set with four inputs and one output. I need to infer the 4 inputs given an output. What is the best way to do this?

I'm assuming that you mean your inputs are 4-dimensional vectors and your outputs are 1 dimensional vectors. If you think that the output vectors give you enough information to completely determine the inputs (at least approximately), then just treat your 1D "outputs" as inputs and your 4D "inputs" as outputs.

Another case is that very different inputs are mapped to similar outputs in your dataset, and you really just want to compute a prototype input for a given output. For example, you might want to produce an image of the letter "a" from only its one-hot label. There are many ways to do this. Here's one that I think is cool:

First, train a neural net the way you normally would with 4D inputs mapping to 1D outputs. Now let's say you're given a 1D output y. Generate some 4D noise x' and pass it through the net to obtain y'. Now pretend that the values in this input are parameters in your network, and hold your actual parameters constant. Use y and y' to compute a loss, and perform gradient descent on your synthetic input. If you repeat this enough times, you should get something that your network thinks deserves the output y.


Should RL rewards diminish over time? Should a reward be cumulative or diminish over time?

For example, say an agent performed a good action at time $t$ and received a positive reward $R$. If reward is cumulative, $R$ is carried on through for the rest of the episode, and summed to any future rewards. However, if $R$ were to diminish over time (say with some scaling $\frac{R}{\sqrt{t}}$), then wouldn't that encourage the agent to keep taking actions to increasing its reward?

With cumulative rewards, the reward can both increase and decrease depending on the agents actions. But if the agent receives one good reward $R$ and then does nothing for a long time, it still has the original reward it received (encouraging it to do less?). However, if rewards diminish over time, in theory that would encourage the agent to keep taking actions to maximise rewards.

I found that for certain applications and certain hyperparameters, if reward is cumulative, the agent simply takes a good action at the beginning of the episode, and then is happy to do nothing for the rest of the episode (because it still has a reward of $R$).

RL agents - implemented correctly - do not take previous rewards into account when making decisions. For instance value functions only assess potential future reward. The state value or expected return (aka utility) $G$ from a starting state $s$ may be defined like this:

$$v(s) = \mathbb{E}_{\pi}[G_t|S_t=s] = \mathbb{E}_{\pi}[\sum_{k=0}^{\infty} \gamma^kR_{t+k+1}|S_t=s] $$

Where $R_t$ is the reward distribution at time $t$, and $\mathbb{E}_{\pi}$ stands for
expected value given following the policy $\pi$ for action selection.

There are a few variations of this, depending on setting and which value function you are interested in. However, all value functions used in RL look at future sums of reward from the decision point when the action is taken. Past rewards are not taken into account.

An agent may still select to take an early high reward over a longer term reward, if:

The choice between two rewards is exclusive
The return is higher for the early reward. This may depend on the discounting factor, $\gamma$, where low values make the agent prefer more immediate rewards.

If your problem is that an agent selects a low early reward when it could ignore it in favour of something larger later, then you should check the discount factor you are using. If you want a RL agent to take a long term view, then the discount factor needs to be close to $1.0$.

The premise of your question however is that somehow a RL agent would become "lazy" or "complacent" because it already had enough reward. That is not an issue that occurs in RL due to the way that it is formulated. Not only are past rewards not accounted for when calculating return values from states, but there is also no formula in RL for an agent receiving "enough" total reward like a creature satisfying its hunger - the maximisation is applied always in all states.

There is no need to somehow decay past rewards in any memory structure, and in fact no real way to do this, as there is no data structure that accumulates past rewards used by any RL agent. You may still collect this information for displaying results or analysing performance, but the agent doesn't ever use $r_{t}$ to figure out what $a_t$ should be.

  I found that for certain applications and certain hyperparameters, if reward is cumulative, the agent simply takes a good action at the beginning of the episode, and then is happy to do nothing for the rest of the episode (because it still has a reward of R

You have probably formulated the reward function incorrectly for your problem in that case. A cumulative reward scheme (where an agent receives reward $a$ at $t=1$ then $a+b$ on $t=2$ then $a+b+c$ on $t=3$ etc) would be quite specialist and you have likely misunderstood how to represent the agent's goals. I suggest ask a separate question about your specific environment and your proposed reward scheme if you cannot resolve this.


What are the differences between TensorFlow and PyTorch? What are the differences between TensorFlow and PyTorch, both in terms of performance and functionality?

TensorFlow was developed by Google and is based on Theano (Python library), while Facebook developed PyTorch using the Torch library. Both frames are useful and have a great community behind them. Both provide machine learning libraries to accomplish various tasks and do the job. TensorFlow is a powerful and deep learning tool with active visualization and debugging capabilities. TensorFlow also offers serialization benefits since the entire graphic is saved as a protocol buffer. It also has support for mobile platforms and offers a production-ready implementation. PyTorch, on the other hand, is still gaining momentum and attracting Python developers, since it is more Python friendly. In summary, TensorFlow is used to speed things up and create AI-related products, while research-oriented developers prefer PyTorch.


Is MIRI doing genuine high-quality research? Recently, I found out about somewhat famous Eliezer Yudkowsky and Machine Intelligence Research Institute he founded. Their philosophy and organisation seem interesting but I'm curious about their credibility. I'm pretty sure this is not a con and they seem to be producing a lot of articles. However, few of those are published and none in the journals mentioned here.

So, is MIRI doing genuine high-quality research?

Whether or not it's produced good research is a question I can't answer.  But I find it interesting and it seems reliable.

For instance, this paper on Parametric Bounded Löb's Theorem and Robust Cooperation of Bounded Agent is drawn from Cornell University's site.  Likewise this recent paper on Logical Induction.

That said, they also seem to publish papers that have not been subject to peer review, which is an issue not specific to the site, and seems to more widespread phenomenon per easy access to digital files in general.

In relation to MIRI specifically, possibly this is a function of an an applied field advancing so rapidly that publication in journals is not seen as a priority, or even particularly relevant.


Is the expected value we sample in TD-learning action-value Q or state-value V? Both MC and TD are model-free and they both follow a sample trajectory (in the case of TD, the trajectory is cut-short) to estimate the return (we basically are sampling Q values). Other than that, the underlying structure of both algorithms is exactly the same. However, from the blogs and texts I read, the equations are expressed in terms of V and NOT Q. Why is that?

However, from the blogs and texts I read, the equations are expressed in terms of V and NOT Q. Why is that?

MC and TD are methods for associating value estimates to time step based on experienced gained in later time steps. It does not matter what kind of value estimate is being associated across time, because all value functions are expressing the same thing in general, which is the expected return conditioned on a "current position" within the MDP. In MC the association is directly with a sampled return, in TD with a sampled combination of immediate reward and a later value estimate - most commonly in TD the same kind of value estimate (e.g. matching later state value estimates to state values).
Both approaches can be analysed and used from the perspective of both state value (V) and action value (Q) functions. They also apply to other value functions - e.g. afterstate values.
It is quite common for textbooks and tutorials to use the slightly simpler state value function to explain how MC or TD learning work in general, outside of being used for any purpose. You can also use the state value function for model-free policy evaluation in MC and TD.
However, without a model, you cannot use state value function for control (i.e. to learn an optimal policy). To pick the best action using state values, you need to do something like this:
$$\pi(s) = \text{argmax}_a [ \sum_{r,s'} p(r,s'|s,a)(r + \gamma v(s'))]$$
The problem here is that $p(r,s'|s,a)$ is a model of the environment. So, if it is needed, the control method would not be model-free.
Hence when you learn about MC or TD in a control scenario, model-free methods to learn optimal policies, then you generally need to use an action value (sometimes you can use an afterstate value, if the action involves choosing the next state directly).
With an action value function, the greedy policy becomes:
$$\pi(s) = \text{argmax}_a q(s, a)$$
This does not refer to any model of the environment. So it can be used when you have none.


Can a neuron have both a bias and a threshold? I have not seen a neuron that uses both a bias and a threshold. Why is this?

I assume you're talking about a perceptron threshold function. One definition of it with an explicit threshold is
$$f(\textbf{x})=
\begin{cases}
1& \text{if } \textbf{w}\cdot\textbf{x} > t\\
0& \text{otherwise}
\end{cases}.$$

Another form with a bias is
$$f(\textbf{x})=
\begin{cases}
1& \text{if } \textbf{w}\cdot\textbf{x} + b > 0\\
0& \text{otherwise}
\end{cases}.$$

But these forms are of course equivalent if you set $b=-t$.

There's nothing stopping you from using a perceptron definition with both a bias and a threshold:
$$f(\textbf{x})=
\begin{cases}
1& \text{if } \textbf{w}\cdot\textbf{x} +b > t\\
0& \text{otherwise}
\end{cases}.$$

But this is also equivalent to the other two forms. We can rewrite this as
$$f(\textbf{x})=
\begin{cases}
1& \text{if } \textbf{w}\cdot\textbf{x} > t'\\
0& \text{otherwise}
\end{cases}$$
where $t'=t-b$ is the new threshold. Or, we could rewrite it as
$$f(\textbf{x})=
\begin{cases}
1& \text{if } \textbf{w}\cdot\textbf{x} + a> 0\\
0& \text{otherwise}
\end{cases}$$
where $a=b-t$ is the new bias.

You never see definitions of this function with both a threshold and a bias because it has simpler forms.


Is AI programming useful in everyday programs? I'm curious about Artificial Intelligence. In my everyday job I develop standard applications, like websites with basic functionalities like user subscription, file upload, forms saved in a database...

I mainly know of AI being used in games or robotics fields. But can it be useful in "standard" application development?

Yes, but probably only to a limited degree in the near term.

Where people draw the boundaries around 'artificial intelligence' is fuzzy, but if one takes the broad view, where it incorporates any sort of coding of explicitly cognitive functions, then many routine economic tasks can benefit from artificial intelligence. Many search engines, for example, can be seen as offering artificial intelligence applications as a service.

For more 'standard' applications, most near-team applications of AI have to deal with fraud detection and prevention. If you track a user's cursor moving across the screen, for example, you can build a model that differentiates between humans and bots, and treat the two separately. See this article for an example.

In the longer term, of course, a program that could write programs could write these sort of applications like any other.


Why can neural networks generalize at all? Neural networks are incredibly good at learning functions. We know by the universal approximation theorem that, theoretically, they can take the form of almost any function - and in practice, they seem particularly apt at learning the right parameters. However, something we often have to combat when training neural networks is overtfitting - reproducing the training data and not generalizing to a validation set. The solution to overfitting is usually to simply add more data, with the rationalization that at a certain point the neural network pretty much has no choice but to learn the correct function.

But this never made much sense to me. There is no reason, in terms of loss, that a neural network should prefer a function that generalizes well (i.e. the function you are looking for) over a function that does incredibly well on the training data and fails miserably everywhere else. In fact, there is usually a loss advantage to overfitting. Equally, there is an infinite number of functions that fit the training data and have no success on anything but.

So why is it that neural networks almost always (especially for simpler data) stumble upon the function we want, as opposed to one of the infinite other options? Why is it that neural networks are good at generalizing, when there is no incentive for them to?

A neural network is composed of continuous functions. Neural networks are regularized by adding an l2 penalty on the weights to the loss function. This means the neural network will try to make the weights as small as possible. The weights are also initiallized with a N(0, 1) distribution so the initial weights will also tend to be small. All of this means that neural networks will compute a continuous function that is as smooth as possible while still fitting the data. By smooth I mean that similar inputs will tend to have similar outputs when run through the neural network. More formally, $||x-y||$ small implies $||f(x)-f(y)||$ small where f represents the output from the neural network. This mean that if a neural network sees an novel input $x$ that is close to an input from the training data $y$, then $f(x)$ will tend to be close to $f(y)$. So the end result is that the neural network will classify $x$ based on what the labels for the nearby training examples were. So the neural network is actually a little like k-nearest neighbors in that way.

Another way for neural networks to generalize is using invariance. For example, convolutional neural networks are approximately translation invariant. So this means that if it sees and image where the object in question has been translated then it will still recognize the object.

But it's not giving us the exact function we want. The loss function is a combination of classification accuracy and making the weights small so that you can fit the data with a function that is as smooth as possible. This tends to generalize well for the reasons I said before but it's just an approximation. You can solve the problem more exactly using minimal assumptions with a Gaussian process but Guassian processes are too slow to handle large amounts of data.


Do scientists know what is happening inside artificial neural networks? Do scientists or research experts know from the kitchen what is happening inside complex "deep" neural network with at least millions of connections firing at an instant? Do they understand the process behind this (e.g. what is happening inside and how it works exactly), or it is a subject of debate?

For example this study says:

  However there is no clear understanding of why they perform so well, or how they might be improved.

So does this mean that scientists actually don't know how complex convolutional network models work?

There are many approaches that aim to make a trained neural network more interpretable and less like a "black box", specifically convolutional neural networks that you've mentioned.

Visualizing the activations and layer weights

Activations visualization is the first obvious and straight-forward one. For ReLU networks, the activations usually start out looking relatively blobby and dense, but as the training progresses the activations usually become more sparse (most values are zero) and localized. This sometimes shows what exactly a particular layer is focused on when it sees an image.

Another great work on activations that I'd like to mention is deepvis that shows reaction of every neuron at each layer, including pooling and normalization layers. Here's how they describe it:

  In short, we’ve gathered a few different methods that allow you to
  “triangulate” what feature a neuron has learned, which can help you
  better understand how DNNs work.

The second common strategy is to visualize the weights (filters). These are usually most interpretable on the first CONV layer which is looking directly at the raw pixel data, but it is possible to also show the filter weights deeper in the network. For example, the first layer usually learns gabor-like filters that basically detect edges and blobs.

Occlusion experiments

Here's the idea. Suppose that a ConvNet classifies an image as a dog. How can we be certain that it’s actually picking up on the dog in the image as opposed to some contextual cues from the background or some other miscellaneous object?

One way of investigating which part of the image some classification prediction is coming from is by plotting the probability of the class of interest (e.g. dog class) as a function of the position of an occluder object.
If we iterate over regions of the image, replace it with all zeros and check the classification result, we can build a 2-dimensional heat map of what's most important for the network on a particular image. This approach has been used in Matthew Zeiler’s Visualizing and Understanding Convolutional Networks (that you refer to in your question):

Deconvolution

Another approach is to synthesize an image that causes a particular neuron to fire, basically what the neuron is looking for. The idea is to compute the gradient with respect to the image, instead of the usual gradient with respect to the weights. So you pick a layer, set the gradient there to be all zero except for one for one neuron and backprop to the image.

Deconv actually does something called guided backpropagation to make a nicer looking image, but it's just a detail.

Similar approaches to other neural networks

Highly recommend this post by Andrej Karpathy, in which he plays a lot with Recurrent Neural Networks (RNN). In the end, he applies a similar technique to see what the neurons actually learn:

  The neuron highlighted in this image seems to get very excited about
  URLs and turns off outside of the URLs. The LSTM is likely using this
  neuron to remember if it is inside a URL or not.

Conclusion

I've mentioned only a small fraction of results in this area of research. It's pretty active and new methods that shed light to the neural network inner workings appear each year.

To answer your question, there's always something that scientists don't know yet, but in many cases they have a good picture (literary) of what's going on inside and can answer many particular questions.

To me the quote from your question simply highlights the importance of research of not only accuracy improvement, but the inner structure of the network as well. As Matt Zieler tells in this talk, sometimes a good visualization can lead, in turn, to better accuracy.


Why does the binary cross-entropy work better than categorical cross-entropy in a multi-class single label problem? I was just doing a simple NN example with the fashion MNIST dataset, where I was getting 97% accuracy, when I noticed that I was using Binary cross-entropy instead of categorical cross-entropy by accident. When I switched to categorical cross-entropy, the accuracy dropped to 90%. I then got curious and tried to use binary cross-entropy instead of categorical cross-entropy in my other projects and in all of them the accuracy increased.

Now, I know that binary cross-entropy can be used in a multi-class, multi-label classification problem, but why is working better than categorical cross-entropy in a multiclass single label problem?

https://stats.stackexchange.com/questions/260505/machine-learning-should-i-use-a-categorical-cross-entropy-or-binary-cross-entro
Is relevant.

based on my reading when you have a NN and do Binary crossentropy on what you might call 'Linked category data' the accuracy can tend to be better than in a Categorical crossentropy model. The binary aspect implies the categories can undergo multiple splits before deciding on the exact category when the data is categorically splittable like this in a tree like hierarchy the accuracy can tend to be better.

Think of how difficult it would be to memorize the name of every type of clothing in someones wardrobe if each piece had its own special name. Vs if they had structurally relevant names like Upper/lower for category number one is it warn on your top half or bottom half. followed by inner or outer. It an inner or outer layer of clothing. learning such binary name/feature categories enables a more accurate model. If it was data unrelated in this way it would most likely not be as accurate. The binary model can take advantage learning such features while a muti categorical model I think assumes independence and tries to learn best the features of each group and gives out a prediction of how sure it falls in each category.


Is a GPU always faster than a CPU for training neural networks? Currently, I am working on a few projects that use feedforward neural networks for regression and classification of simple tabular data. I have noticed that training a neural network using TensorFlow-GPU is often slower than training the same network  using TensorFlow-CPU.

Could something be wrong with my setup/code or is it possible that sometimes GPU is slower than CPU?

This changes according to your data and complexity of your models. See following article by microsoft. Their conclusion is

  The results suggest that the throughput from GPU clusters is always
  better than CPU throughput for all models and frameworks proving that
  GPU is the economical choice for inference of deep learning models.
  ...

  It is important to note that, for standard machine learning models
  where number of parameters are not as high as deep learning models,
  CPUs should still be considered as more effective and cost efficient.

Since you are training MLP, it can not be thought as standard machine learning model. See my preprint, The impact of using large training data set KDD99 on classification accuracy. I compare different machine learning algorithms using weka.

As you can see from above image, MLP takes 722 minutes to train while Naive Bayes ~2 minutes. If your data is small and your models parameters are not high, you will see better performance on CPU.


What is the most general definition of "intelligence"? When we talk about artificial intelligence, human intelligence or any other form of intelligence, what do we mean by the term intelligence in a general sense? What would you call intelligent and what not? In other words, how do we define the term intelligence in the most general possible way?

I'm going to preface this answer by noting that persons much smarter than myself have treated this subject in some detail.  That said, as far as I can discern:

When we talk about intelligence we're referring to problem solving strength in relation to a problem, relative to the strength of other intelligences.

This is a somewhat game-theoretic conception, related to rationality and the concept of the rational agent.  Regarding intelligence in this manner may be unavoidable. Specifically, we could define intelligence as the ability to understand a problem or solution or abstract concepts, but we can't validate that understanding without testing it.  (For instance, I might believe I grasp a mathematical technique, but the only way to determine if that belief is real or illusory is to utilize that technique and evaluate the results.)
The reason games like Chess and Go have been used as milestones, aside from longstanding human interest in the games, is that they provide models with simple, fully definable parameters, and, in the case of Go at least, have complexity akin to nature, by which I mean unsolvable/intractable. (Compare to strength at Tic-Tac-Toe, which is trivially solved.)
However, we should consider a point made in this concise answer to a question involving the Turing Test:

"...is [intelligence] defined purely by behaviour in an environment, or by the mechanisms that arrive at that behaviour?"

This is important because Google just gave control over data center cooling to an AI.  Here it is clearly the mechanism itself that demonstrates utility, but if we call that mechanism intelligent, for intelligence to have meaning, we still have to contend with "intelligent how?"  (In what way is it intelligent?) If we want to know "how intelligent?" (its degree of utility) we still have to evaluate its performance in relation to the performance of other mechanisms.
(In the case of the automata controlling the air conditioning at Google, we can say that it is more intelligent than the prior control system, and by how much.)
Because we're starting to talk about more "generalized intelligence", defined here as mechanisms that can be applied to a set of problems, (I include minimax as a form of "axiomatic intelligence" and machine learning as a form "adaptive intelligence"), it may be worthwhile to expand and clarify the definition:

Intelligence is the problem solving strength of a mechanism in relation to a problem or a set of problems, relative to the strength of other mechanisms.

or, if we wanted to be pithy:

Intelligence is as intelligence does (and how well.)


What is a probability distribution in machine learning? If we were learning or working in the machine learning field, then we frequently come across the term "probability distribution". I know what probability, conditional probability, and probability distribution/density in math mean, but what is its meaning in machine learning?
Take this example where $x$ is an element of $D$, which is a dataset,
$$x \in D$$
Let's say our dataset ($D$) is MNIST with about 70,000 images, so then $x$ becomes any image of those 70,000 images.
In many papers and web articles, these terms are often denoted as probability distributions
$$p(x)$$
or
$$p\left(z \mid x \right)$$

What does $p(\cdot)$ even mean, and what kind of output does $p(\cdot)$ give?
Is the output of $p(\cdot)$ a scalar, vector, or matrix?
If the output is vector or matrix, then will the sum of all elements of this vector/matrix always be $1$?

This is my understanding,
$p(\cdot)$ is a function which maps the real distribution of the whole dataset $D$. Then  $p(x)$ gives a scalar probability value given $x$, which is calculated from real distribution $p(\cdot)$. Similar to $p(H)=0.5$ in a coin toss experiment $D={\{H,T}\}$.
$p\left(z \mid x \right)$ is another function that maps the real distribution of the whole dataset to a vector $z$ given an input $x$ and the $z$ vector is a probability distribution that sums to $1$.
Are my assumptions correct?
An example would be a VAE's data generation process, which is represented in this equation
$$p_\theta(\mathbf{x}^{(i)}) = \int p_\theta(\mathbf{x}^{(i)}\vert\mathbf{z}) p_\theta(\mathbf{z}) d\mathbf{z}$$

Random variables
You do not necessarily need to understand the concept of a random variable (r.v.) to understand the concept of a probability distribution, but the concept of a random variable is strictly connected to the concept of a probability distribution (given that each random variable has an associated probability distribution), so, before proceeding, you should get familiar with the concept of an r.v., which is a (measurable) function from the sample space (the set of possible outcomes of an experiment) to a measurable space (you can ignore the definition of a measurable space and assume that the codomain of the random variable is a finite set of numbers).
Probability measure, cdf, pdf and pmf
The expression "probability distribution" can be ambiguous because it can be used to refer to different (even though related) mathematical concepts, such as probability measure, cumulative distribution function (c.d.f.), probability density function (p.d.f.), probability mass function (p.m.f.). If a person uses the expression "probability distribution", he (or she) intentionally (or not) refers to one or more of these mathematical concepts, depending on the context. However, a probability distribution is almost always a synonym for probability measure or c.d.f..
For example, if I say "Consider the Gaussian probability distribution", in that case, I could be referring to either the c.d.f. or the p.d.f. (or both) of the Gaussian distribution. Why couldn't I be referring to the p.m.f. of the Gaussian distribution? Because the Gaussian distribution is a continuous distribution, so it is a distribution associated with a continuous random variable, that is, a random variable that can take on continuous values (e.g. real numbers), so a Gaussian distribution does not have an associated p.m.f. or, in other words, no p.m.f. is defined for the Gaussian distribution. Why don't I simply say "Consider the p.d.f. of the Gaussian distribution." or "Consider a Gaussian p.d.f."? Because it is unnecessarily restrictive, given that, if I say "Consider the Gaussian distribution" I am implicitly also considering a p.d.f. and c.d.f. of the Gaussian distribution.
Similarly, in the case of a discrete distribution, such as the Bernoulli distribution, only the c.d.f. and p.m.f. are defined, so the Bernoulli distribution does not have an associated p.d.f.
However, it is important to recall that both continuous and discrete distributions have an associated c.d.f., so the expression "probability distribution" almost always (implicitly) refers to a c.d.f., which is defined based on a probability measure (as stated above).
Notation
In the same vein, the notation $p(x)$ can be as ambiguous as the expression "probability distribution", given that it can refer to different (but again related) concepts. However, $p(x)$ usually refers to a probability measure (so it refers to a probability distribution, given that a probability distribution is almost always a synonym for probability measure). In this case, assuming for simplicity that the r.v. is discrete, $p(x)$ is a shorthand for $p(X=x)$, which is also written as $\mathbb{P}(X=x)$ or $\operatorname{Pr}(X=x)$, where $X$ is a r.v., $x$ a realization of $X$ (that is, a value that the r.v. $X$ can take) and $X=x$ represents an event. Given that an r.v. is a function, the notation $X=x$ may look a bit weird.
In the case of a discrete r.v., $p(x)$ can also refer to a p.m.f. and it can be defined as $p_X(x) = \mathbb{P}(X=x)$ (I added the subscript $X$ to $p$ to emphasize that this is the p.m.f. of the discrete r.v. $X$). In the case of a continuous r.v., the p.d.f. is often denoted as $f$. In the case of both discrete and continuous r.v.s, the c.d.f is usually denoted with $F$ and it is defined as $F_X(x) = \mathbb{P}(X \leq x)$, where $\mathbb{P}$ is again a probability measure (or probability distribution). The p.d.f. of a continuous r.v. is then defined as the derivative of $F$. At this point, it should be clear why a probability distribution can refer to different but related concepts, but, in any case, it always refers to a probability measure.
Empirical distributions
There are also empirical distributions, which are distributions of the data that you have collected. For example, if you toss a coin 10 times, you will collect the results ("heads" or "tails"). You can count the number of times the coin landed on heads and tails, then you plot these numbers as a histogram, which essentially represents your empirical distribution, where the adjective "empirical" usually refers to the fact that there is an experiment involved.
Multivariate r.v.s and distributions
To complicate things even more, there are also multivariate random variables and probability distributions. However, all the concepts above more or less are also applicable in this case.
Parametrized distributions
A parametrized probability distribution, often denoted by $p_{\theta}$, is
a family of probability distributions (defined by the parameters $\theta$), rather than a single probability distribution. For example, $\mathcal{N}(0, 1)$ refers to a single Gaussian distribution with zero mean and unit variance. However, $\mathcal{N}(\mu, \sigma)$, where $\theta=(\mu,  \sigma)$ is a variable, is a family (or collection) of distributions.
Conclusion
To conclude, it is completely understandable that you are confused, given that the terminology and notation are used inconsistently, and there are several involved concepts, which I have not extensively covered in this answer (for example, I have not mentioned the concept of a probability space). If you get familiar with the concepts of probability measures, random variables, p.m.f., p.d.f., c.d.f., etc., and how they are related, then you will start to get a better feeling of the whole picture.


Is there any other rotated object detection datasets? I have googled for a long time for rotated object detection datasets. Most of papers focused on rotated object detection using DOTA, HRSC2016 or coco text detection dataset. Some researcher also collect their own datasets but almost all of their theme is areal object detection. Is there any others dataset focus on rotated object detection?

Multiscale Rotated Bounding Box-Based Deep Learning Method
Here's a link for a reference


Why do neural networks trained on identical datasets and with identical hyper-parameters have different performances? I found that fully connected neural networks trained on identical data sets with identical hyper-parameters can have different performances or accuracies (7-8% of deviation). Is this an unusual phenomenon? What is the cause? What is an acceptable deviation?

This should simply be due to the fact that network weight initialization is random.


What is the meaning of the square brackets in ant colony optimization? I'm studying the paper "Minimizing Total Tardiness on a Single Machine Using Ant Colony Optimization" which has proposed to use Ant colony optimization to SMTWTP.
According to this paper:

Each artificial ant iteratively and independently decides which job to
append to the sub-sequence generated so far until all jobs are
scheduled, Each ant generates a complete solution by selecting a job $j$
to be on the $i$-th position of the sequence. This selection process is
influenced through problem-specific heuristic information called
visibility and denoted by $\eta_{ij}$ as well as pheromone trails denoted by $\tau_{ij}$. The former is an indicator of how good the choice of that job
seems to be and the latter indicates how good the choice of that job
was in former runs. Both matrices are only two dimensional as a
consequence of the reduction in complexity

They have proposed this formula for the probability that job $j$ be selected to be processed on position $i$ (page 9 of the linked paper):
$$
\mathcal{P}_{i j}=\left\{\begin{array}{cl}
\frac{\left[\tau_{i j}\right]^{\alpha}\left[\eta_{i j}\right]^{\beta}}{\sum_{h \in \Omega}\left[\tau_{i h}\right]^{\alpha}\left[\eta_{i h}\right]^{\beta}} & \text { if } j \in \Omega \\
0 & \text { otherwise }
\end{array}\right.\tag{1}\label{1}
$$
but I can't understand what $[]$ surrounding $\eta_{ij}$ and $\tau_{ij}$ indicates. Does it show that these values are matrices?

The square brackets $[]$ in $[\tau_{ij}]^\alpha$ and $[\eta_{ij}]^\beta$ may be just a way of emphasing that the elements $\tau_{ij} \in \mathbb{R}$ and $\eta_{ij} \in \mathbb{R}$ of respectively the matrices $\mathbf{\tau} \in \mathbb{R}^{n \times n}$ and $\mathbf{\eta} \in \mathbb{R}^{n \times n}$ (where $n$ is the number of nodes in the graph) are respectively raised to $\alpha$ and $\beta$, so they could have used also other type of brackets, for example, $()$. It may also be a way of indicating that $[\tau_{ij}]$ and $[\eta_{ij}]$ are $1 \times 1$ matrices or vectors that contain respectively the scalars $\tau_{ij}$ and $\eta_{ij}$, so you are multiplying matrices or vectors (dot product).

This notation is also used in the paper that introduced the ant colony system (ACS) (and it is probably used in many other papers related to ant colony optimization). See equation 1 of Ant Colony System: A Cooperative Learning Approach to the Traveling Salesman Problem (1997) by Dorigo and Gambardella.


What is "early stopping" in machine learning? What is early stopping in machine learning and, in general, artificial intelligence? What are the advantages of using this method? How does it help exactly?

I'd be interested in perspectives and links to recent research.

In some iterative learning methods the more iterations you apply the more specific your model becomes about the training set. If there are too many iterations, your model will become too specifically trained for the training samples and will score less on other samples that are not seen during the training phase. This is call over-fitting, though over-fitting is not specific to iterative learning methods.

One solution to prevent over-fitting in these iterative learning algorithms is early stopping. Normally a control group of samples called validation samples (validation set) are used to validate the model and notify when it starts to over-fit. The validation set is not used by the training algorithm, however, its corresponding outputs are known and after each iteration, its samples are employed to measure how good the model currently works. As soon as the performance on the validation set stops to grow and starts to drop we stop iterating the training algorithm. This is called early stopping which can help to maximize the generalization power of our learned model.

Note that if we use the training set itself for validation the performance will always increase because that is what the learning algorithm is designed to do. However, the learning algorithm does not know how specifically it should learn the training set and that is why we need methods like early stopping.


Game AI evaluation function and making progress towards winning In two-player games, the exact value of the evaluation function doesn't matter, as long as it's bigger for better positions. However, for learning, it's customary when it does change when the best move gets made. This way, the learning can minimize the difference between the directly computed value $f(0, p)$ of a position $p$ and the value obtained from $n$ step minimax $f(n, p)$.
What I'm missing here is a way to direct the evaluation function to actually winning. For example, a perfect evaluation function for a won position in chess would always return $+1$ without any hint on how to progress towards a checkmate. In a chess variant without the fifty-move limit, it could play useless turns forever.
I guess this is a rather theoretical problem, as we won't ever have such a good function, but I wonder if there's a way to avoid it?

What I'm missing here is a way to direct the evaluation function to actually winning. For example, a perfect evaluation function for a won position in chess would always return +1 without any hint how to progress towards checkmate. In a chess variant without the fifty-move limit, it could play useless turns forever.

  I guess, this is a rather theoretical problem as we won't ever have such a good function, but I wonder if there's a way to avoid it?

It certainly isn't just a theoretical problem, it can occur in practice too. For "large" games this problem won't occur in the early game, but it can start occurring in the later stages of a game when terminal positions can actually be reached directly through exhaustive searches. It can also occur right from the beginning in extremely small / simple games, like Tic Tac Toe.

Addressing this is not (necessarily) just a matter of defining the evaluation function alone, it can also depend on which search or learning algorithm is used. So, I'll consider a few different cases.

Case 1: Minimax / Alpha-Beta / other similar "exhaustive" searches

When using Minimax / Alpha-Beta / other search algorithms based on those, the easiest solution to the problem you describe is to use iterative deepening. As soon as you prove a win for yourself at a certain depth level d using iterative deepening, you can simply stop the search, don't search if there are any other wins to be proven at depth d + 1, just play along the line you've just proven to be a winning line. This way, you will always go for the win in the lowest number of moves.

A related advantage of this approach is that, as soon as you prove a loss at depth d for yourself, you can also cancel the search process and play the move that was best according to your evaluations at depth d - 1. This will often be your best chance to still grab a win in cases where your opponent fails to see their win at depth d.

Case 2: Monte-Carlo Tree Search / other searches with randomness

Monte-Carlo Tree Search is a well-known search algorithm that incorporates an element of randomness in its search. With these kinds of algorithms, the problem you describe tends not to be a real issue. Due to the randomness in the search, wins that can be achieved in a small number of moves tend to be evaluated better than longer-distance wins in practice. In long-distance wins, there is a greater chance that the randomness in the search process causes an incorrect move to be played somewhere along the long-distance win, which reduces the evaluation of such a line of play.

Case 3: (Reinforcement) Learning approaches

These approaches tend to involve some element of randomness due to the need for exploration in learning, which leads to similar reasoning as described for MCTS above. Also, in Reinforcement Learning, we typically use a discount factor gamma < 1.0 (e.g. gamma = 0.99) which causes distant rewards to be viewed as less important than close rewards, even if we don't do such discounting for the final evaluation of the performance of an algorithm. See, for example, a lot of the work on Atari games (DeepMind's DQN, etc.). Algorithms are evaluated according to their undiscounted scores, but learning still uses a bit of discounting because, in practice, this is found to be beneficial for learning.


Why is the derivative of the softmax layer shaped differently than the derivative of other neurons? If the derivative is supposed to give the rate of change of a function at that point, then why is the derivative of the softmax layer (a vector) the Jacobian matrix, which has a different shape than the output/softmax vector? Why is the shape of the softmax vector's derivative (the Jacobian) different than the shape of the derivative of the other activation functions, such as the ReLU and sigmoid?

When you use the softmax activation function is usually as a last layer of your network and to get an output that is a vector. Now your confusion is about shapes, so let's review a bit of calculus.
If you have a function $$f:\mathbb{R}\rightarrow\mathbb{R}$$
the derivative is a function on its own and you have $$f':\mathbb{R}\rightarrow\mathbb{R}.$$
If you increase the dimension of the input space have
$$f:\mathbb{R}^n\rightarrow\mathbb{R}.$$
The "derivative" in this case is called gradient and it is a vector collecting the $n$ partial derivatives of $f$. The input space of the gradient function is $\mathbb{R}^n$ (the same as for $f$), but the output is the collection of the $n$ derivatives, so the output space is also $\mathbb{R}^n$. In other words
$$\nabla f:\mathbb{R}^n\rightarrow\mathbb{R}^n,$$
which makes sense as for each point $x$ of the input space you get a vector ($\nabla f(x)$) as output.
So far so good, but what happens if you consider a function that takes a vector as input and spits out a vector as output, i.e.
$$f:\mathbb{R}^n\rightarrow\mathbb{R}^m?$$
How to compute the equivalent of the derivative? (This is the softmax case, where you have a vector as input and a vector as output.)
You can reduce this case to the previous case considering $f=(f_1, \dots, f_m)$, where $f_i:\mathbb{R}^n\rightarrow\mathbb{R}.$ Now for each $f_i$ you can compute the gradient $\nabla f_i$ and end up with $m$ gradients. When you evaluate them at a point $x\in\mathbb{R}^n$ you get $m$ $n-$dimensional vectors. These vector can be collected in a matrix, which is the Jacobian; formally
$$Jf:\mathbb{R}^n\rightarrow\mathbb{R}^{m\times n}.$$
Finally, to answer your question, you get a Jacobian "instead"  of a gradient (they all represent the same concept) because the output of the softmax is not a single number but a vector.
By the way the sigmoid and relu are functions with one dimensional input and output, so they don't really have a gradient but a derivative. The trick is that people write $\sigma(W)$, where $W$ is vector or a matrix, but they mean that $\sigma$ is applied component-wise, as a function from $\mathbb{R}$ to $\mathbb{R}$ (I know, it's confusing).
(I know, I kind of skipped your other question, but this answer was already long and I think you can convince yourself that the dimensions match correctly (with the Jacobian) for the update rule to work. If not I'll edit)


What assumptions are made when positing the emergence of superintelligence? Many experts seem to think that artificial general intelligence, or AGI, (on the level of humans) is possible and likely to emerge in the near-ish future. Some make the further step to say that superintelligence (much above the level of AGI) will appear soon after, through mechanisms like recursive self-improvement from AGI (from a survey).

However, other sources say that such superintelligence is unlikely or impossible (example, example).

What assumptions do those who believe in superintelligence make? The emergence of superintelligence has generally been regarded as something low-probability but possible (e.g. here). However, I can't seem to find an in-depth analysis of what assumptions are made when positing the emergence of superintelligence. What specific assumptions do those who believe in the emergence of superintelligence make that are unlikely, and what have those who believe in the guaranteed emergence of superintelligence gotten wrong?

If the emergence of superintelligence is to be seen as a low-probability event in the future (on par with asteroid strikes, etc.), which seems to be the dominant view and is the most plausible, what assumptions exactly makes it low-probability?

I am not an expert on the topic, but I will provide some information that could be useful or helpful.

I think that the first and maybe trivial assumption that people make when they say that AGI or SI can emerge is that general intelligence (whatever the definition is) is computable, i.e. there's a Turing machine or, in general, a mathematical model of computation that can simulate an algorithm that possibly represents general intelligence. I don't know how likely the mind of a human (or any other general intelligence) is computable (or to what extent), but there's the so-called computational theory of mind (CTM) that goes into this direction.

An assumption related to the emergence of SI is that recursive self-improvement is physically possible. I also don't know exactly how likely or unlikely recursive self-improvement is, but my limited knowledge of thermodynamics (and physics) suggests that it will not be possible (at least, at the pace or the way people claim or want to suggest). I am not saying that we will not see more progress in the next years (we will), but this doesn't mean that we will be able to create an AGI. People often overestimate themselves and their intelligence.

A third assumption is related to the current achievements in the artificial intelligence field. Many people claim that there has been a lot of progress in recent years and that there's no reason to believe that this will not continue. However, history tells us that the predictions of AI scientists about the future of the AI field were often wrong. There have been at least 2 AI winters because of these wrong predictions and unmet expectations.

Many people also don't exclude the possibility of the emergence of AGI or SI only because they have not yet been proven wrong.

Another assumption is that there won't be any catastrophe (like COVID-19  can potentially be) that will significantly slow down the general scientific progress.


Imposing physical constraints (previous knowledge) in a neural network for regression I'm trying to train a neural network to do a multiple non-linear regression $y=f(x_i), i=1,2…N$. So far it works good (low MSE), but some predictions $y$ are “non-physical”, for instance for our application it is known  from first principles that when $x_2$ increases, then $y$ also has to increase ($dy/dx_2>0$), but in some instances the neural network’s output doesn’t comply with this constraint. Another example is that $y + x_5 + x_7$ should be less than a constant $K$

I thought about adding a penalty term to the loss function to enforce these constraints, but I am wondering if there is a "harder" way to impose such a constraint (that is, to ensure that these constraints will always hold, no only that non-physical predictions will be penalized)

Some ideas out the top of my head:

In the case of $dy/dx_2>0$ you could compute the gradient using the chain rule and limit the weights so that the constrain holds
In the case of $y + x_5 + x_7 < K$ you could use a clipping function on the output layer?


Is it required that taking an action updates the state? For some environments taking an action may not update the environment state. For example, a trading RL agent may take an action to buy shares s. The state at time t which is the time of investing is represented as the interval of 5 previous prices of s. At t+1 the share price has changed but it may not be as a result of the action taken. Does this affect RL learning, if so how ? Is it required that state is updated as a result of taking actions for agent learning to occur ?

In gaming environments it is clear how actions affect the environment. Can some rules of RL breakdown if no "noticeable" environment change takes place as a result of actions ?

Update:

"actions influence the state transitions", is my understanding correct:
If transitioning to a new state is governed by epsilon greedy and epsilon is set to .1 then with .1 probability the agent will choose an action from the q table which has max reward reward for the given state. Otherwise the agent randomly chooses and performs an action then updates the q table with discounted reward received from the environment for the given action.

I've not explicitly modeled an MDP and just defined the environment and let the agent determine best actions over multiple episodes of choosing either a random action or the best action for the given state, the selection is governed by epsilon greedy.

But perhaps I've not understood something fundamental in RL. I'm ignoring MDP in large part as I'm not modeling the environment explicitly. I don't set the probabilities of moving from each state to other states.

A very vague question. What's the objective?

Reinforcement Learning (RL) typically uses the Markov Decision Process framework, which is a sequential decision making framework. In this framework, actions influence the state transitions. In other words, RL deals with controlling (via actions) a Markov chain. The objective in RL is figure out how to take actions in an optimal (in some sense) way!

If, in the application you mentioned, the actions don't influence the state transitions and the objective is to predict states, RL is not required. It's just a regression/ time-series problem.


Can BERT be used for sentence generating tasks? I am a new learner in NLP. I am interested in the sentence generating task. As far as I am concerned, one state-of-the-art method is the CharRNN, which uses RNN to generate a sequence of words.

However, BERT has come out several weeks ago and is very powerful. Therefore, I am wondering whether this task can also be done with the help of BERT? I am a new learner in this field, and thank you for any advice!

For newbies, NO.

Sentence generation requires sampling from a language model, which gives the probability distribution of the next word given previous contexts. But BERT can't do this due to its bidirectional nature.

For advanced researchers, YES.

You can start with a sentence of all [MASK] tokens, and generate words one by one in arbitrary order (instead of the common left-to-right chain decomposition). Though the text generation quality is hard to control.

Here's the technical report BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model, its errata and the source code.

In summary:

If you would like to do some research in the area of decoding with
BERT, there is a huge space to explore
If you would like to generate
high quality texts, personally I recommend you to check GPT-2.


Is binary classification using CNN possible if the training data only consists of one class? Is binary classification using CNN possible if the training data only consists of one class?
I am working on landslide risk assessment using Convolutional Neural Networks and I want to train a network that can recognize high-risk areas using multi-spectral imagery. The bands will contain numeric and categorical data that I have found to be related to my field of work.
The problem is that I only have historical data indicating where a landslide has happened before and defining zones as low-risk is not reliable in this field (since we are not yet sure how these variables affect the risk or susceptibility, and I don't want to bias my categorization) and my training data will be made up of only one class.
Can this be done? Is training a network from scratch using only one class of training data possible?
If so, after building this network, can I use it to classify any zone and get any meaningful data from its output for risk assessment (for example, output value "1" being "similar to past landslides" and "0" being "not similar at all")?

It probably won't work because of in the training of the artificial intelligence, it'll set the weights to always answer class 1, and your data will say you're right, and it'll continue forever.