Python Network for Simple Image Classification I'm wondering if there exists a network for simple image classification. What I mean by this is if I have two image datasets, one of horses and one of zebras, I want to train off the horses and classify an image as either a horse or not a horse, so if I test it on an image of a horse, it says it is a horse, but if I use a zebra, it says it is not a horse. Does any library/project for this exist?

Using neural networks

If it is as simple as that, probably a neural network consisting of just a simple convolutional network with a few filters, a fully connected layer with 2 neurons, and a softmax layer, might do it.

To develop that, you can use Keras wrapper over tensorflow or theano, but I think scikit-learn also provides some basic neural network functionalities.

Depending on the simplicity of the images on your dataset, you might do it even with no convolutional layer if there are not many non-linearities, or you might need more complexity (eg. adding convolutional and max pooling layers).

I made personally for teaching a workshop a very simple example of image classification, with 3 groups (trees, people, vehicles), after building a dataset collecting and selectioning images from the CIFAR-100 dataset.  You can see this ultra simple example here. However, it is in spanish. Let me translate the most important piece of code from it:

model = Sequential()

# CONVOLUTION. Final size: 29x29x32
model.add(Conv2D(   32,
                    (4, 4),
                    padding='same',
                    input_shape=data_x_train.shape[1:]
        ))

model.add(Activation('relu'))

# CONVOLUTION. Final size: 26x26x32
model.add(Conv2D(   32,
                    (4, 4),
                    padding='same',
                    input_shape=data_x_train.shape[1:]
        ))

model.add(Activation('relu'))

# MAX POOLING. Final size: 13x13x32
model.add(MaxPooling2D(pool_size=(2,2)))

# Dropout to prevent overfitting
model.add(Dropout(0.25))

# CONVOLUTION. Final size: 10x10x64
model.add(Conv2D(   64,
                    (4, 4),
                    padding='same',
                    input_shape=data_x_train.shape[1:]
        ))

model.add(Activation('relu'))

# MAX POOLING. Final size: 5x5x64
model.add(MaxPooling2D(pool_size=(2,2)))

# Dropout to prevent overfitting
model.add(Dropout(0.25))

# Fully connected layer. Neuron number: 1600
model.add(Flatten())

# Output layer: 3 neurons
model.add(Dense(3))
model.add(Activation('softmax'))

So the input for this model is an array of shape [num_rows, 32, 32]. 32x32 is the image size, and I assume 1 color channel. As I say, this is ultra simple. The output, some kind of probability for each of the 3 classes.

Using other aproaches

However, there are many other supervised learning models that you can use for that, most of them supported by scikit-learn, such as kNN, random forests decision trees, RBF SVM, or even just a kernelized multiclass logistic regression. See here.


Teaching neural network on fixed data Imagine that we have a black box that have 100 binary inputs and 30 binary outputs.

We can generate values for inputs and get relevant set of outputs.

How does one teach a neural network to predict the binary inputs (or list of input values with probabilitys) using the outputs?

Need practice.

The problem you describe is a matter of perspective; although you want to predict the inputs, defining the outputs of the BB (blackbox) as the inputs to a new system (an inverse BB) whose outputs are that of the inputs of the original BB, then one can use previous literature. There are some assumptions using this method such as "each input can be calculated from an inverse function using 1 or more outputs" so take precaution.

I'd recommend reading through this paper by Richard Sutton. Its introduction defines the problem mathematically and references some common approaches as well as its own method.


How would AI be able to self-examine? As I see some cases of machine-learning based artificial intelligence, I often see they make critical mistakes when they face inexperienced situations.

In our case, when we encounter totally new problems, we acknowledge ourselves that we are not skilled enough to do the task and hand it to someone who is capable of doing the task.

Would AI be able to self-examine objectively and determine if it is capable of doing the task?

If so, how would it be accomplished?

Several AI systems will come up with a level of confidence to the solution found. For example, neural networks can indicate how relatable is the input problem to the ones it was trained with. Similarly, genetic algorithms work with evaluation functions, that are used to select best results, but depending on how they're built (the functions), they can indicate how close the algorithms are to an optimal solution.

In this case, the limit to when this is acceptable or not will depend on a threshold set beforehand. Is 50% confidence good enough? Maybe it's ok for OCR apps (spoiler: it's not), but is it for medical applications?

So yes, AI systems do currently have the capacity of determining if they're performing well or not, but how acceptable that is is currently based on the domain of the problem, which currently stands outside of what is built into an AI.


What is meant by "ground truth" in the context AI? What does "ground truth" mean in the context of AI especially in the context of machine learning?
I am a little confused because I have read that the ground truth is the same as a label in supervised learning. And I think that's not quite right. I thought that ground truth refers to a model (or maybe the nature) of a problem. I always considered it as something philosophical (and that's what also the vocabulary 'ground truth' implies), because in ML we often don't build a describing model of the problem (like in classical mechanics) but rather some sort of a simulator that behaves like it is a describing model. That's what we/I call sometimes black box.
What is the correct understanding?

In the context of ML, ground truth refers to information provided by direct observation (empirical evidence). If you're training an algorithm to classify your data, then the ground truth will be the actual, true labels which could for example be manually annotated by an domain expert. Please note that the models prediction or the inferred labels, are not considered ground truth.


What are examples of resources that describe the basics of Spiking Neural Networks in detail? I'm very interested in writing a Spiking Neural Network engine (SNN) from scratch, but I can't find the basic information I need to get started.
For example, I've seen pictures of the individual signals that combine to form a neuron pulse in several research papers, with no information on the equations in use.  It's not the focus of the papers, and the authors assume the readers have that knowledge already.  Some papers reference software that provides this foundation (NEST, pyNN, etc.), but the documentation for the software is similarly light on details.
There is a ton of information out there on the more common network types, but SNNs have not yet made it into the mainstream.
So, where do I get this basic information? Has someone pulled together any recipes/examples/tutorials for an SNN, as has been done with all the other network types?

There is a book Spiking Neuron Models: Single Neurons, Populations, Plasticity (2002) by Wulfram Gerstner and Werner M. Kistler.
I also found these slides that could be useful.
(Additional resources would be appreciated.)


What are the limitations of the hill climbing algorithm and how to overcome them? What are the limitations of the hill climbing algorithm? How can we overcome these limitations?

As @nbro has already said that Hill Climbing is a family of local search algorithms. So, when you said Hill Climbing in the question I have assumed you are talking about the standard hill climbing. The standard version of hill climb has some limitations and often gets stuck in the following scenario:

Local Maxima: Hill-climbing algorithm reaching on the vicinity a local maximum value, gets drawn towards the peak and gets stuck there, having no other place to go.
Ridges: These are sequences of local maxima, making it difficult for the algorithm to navigate.
Plateaux: This is a flat state-space region. As there is no uphill to go, algorithm often gets lost in the plateau.

To resolve these issues many variants of hill climb algorithms have been developed. These are most commonly used:

Stochastic Hill Climbing selects at random from the uphill moves. The probability of selection varies with the steepness of the uphill move.
First-Choice Climbing implements the above one by generating successors randomly until a better one is found.
Random-restart hill climbing searches from randomly generated initial moves until the goal state is reached.

The success of hill climb algorithms depends on the architecture of the state-space landscape. Whenever there are few maxima and plateaux the variants of hill climb searching algorithms work very fine. But in real-world problems have a landscape that looks more like a widely scattered family of balding porcupines on a flat floor, with miniature porcupines living on the tip of each porcupine needle (as described in the 4th Chapter of the book Artificial Intelligence: A Modern Approach). NP-Hard problems typically have an exponential number of local maxima to get stuck on.
Given algorithms have been developed to overcome these kinds of issues:

Stimulated Annealing
Local Beam Search
Genetic Algorithms

Reference Book - Artificial Intelligence: A Modern Approach


Q-Learning the generic maze solution After doing some exercices on Q-learning for maze solving, I wondered : my q-learning algorithms solve only ONE maze. The AI doesn't learn how to solve mazes, so how can I achieve it ?

For instance learn "follow one wall until the end of the maze" instead of "turn left left right..." ?

I guess it is the same approach for self driving cars ?

I'm going to assume here that you're using the standard, basic, simple variant of $Q$-learning that can be described as tabular $Q$-learning, where all of your state-action pairs for which you're learning $Q(s, a)$ values are represented in a tabular fashion. For example, if you have 4 actions, your $Q(s, a)$ values are likely represented by 4 matrices (corresponding to the 4 actions), where every matrix has the same dimensionality as your maze (I'm assuming that your maze is a grid of discrete cells here).

With such an approach, you are learning $Q$ values separately for every single individual state (+ action). Such learned values will always only be valid for one particular maze (the one you have been training in), as you seem to have already noticed. This is a direct consequence of the fact that you're learning individual values for specific state-action pairs. The things you are learning ($Q$ values) can therefore not directly be transferred to a different maze; those particular states from the first maze do not even exist in the second maze!

Better results may be achievable with different state representations. For example, instead of representing states by their coordinates in a grid (as you would likely do with a tabular approach), you'd want to describe states in a more general way. For example, a state could be described by features such as:

Is there a wall right in front of me?
Is there a wall immediately to my right?
Is there a wall immediately to my left?

An alternative that also may actually be able to better generalize to some extent could be pixel-based inputs if you have images (a top-down image or even a first-person-view).

When states are represented by such features, you can no longer use the tabular RL algorithms that you are likely familiar with though. You'd have to use function approximation instead. With good state-representations, those techniques may have a chance of generalizing. You'd probably want to make sure to actually also use a variety of different mazes during the training process though, otherwise they'd likely still overfit to only a single maze used in training.


Accuracy scores in a Deep Learning project I'm using three pre-trained deep learning models to detect vehicles and count from an image data set. The vehicles belong to one of these classes ['car', 'truck', 'motorcycle', 'bus']. So, for a sample I have manually counted number of vehicles in each image. Also, I employed the three deep learning models and obtained the vehicle counts. For example:

    Actual        | model 1 count| model 2 count  | model 3 count
------------------------------------------------------------------
    4 cars, 1 bus | 2 cars       | 2 cars, 1 truck| 4 cars
    2 cars        | 0            | 1 truck        | 1 car, 1 bus

In this case, how can I measure accuracy scores such as precision and recall?

Precision is the number of true positives over the number of predicted positives(PP), and recall is the number of true positives(TP) over the number of actual positives(AP) you get. I used the initials just to make it easier ahead.

A true positive is when you predict a car in a place and there is a car in that place.

A predicted positive is every car you predict, being right or wrong does not matter.

A actual positive is every car that actually is in the picture.

You should calculate these values separately for each category, and then sum over the examples you sampled, if I am not mistaken.

So for the CAR category you have (assuming the predictions do match with the target, i.e., you are not predicting a truck as a car for example) :

model 1
line 1  -> 2 TP, 2 PP, 4 AP
line 2  -> 0 TP, 0 PP, 2 AP

So in total precision  is 2/2 = 1 and recall is 2/6 = 0.3(3).

You can then do the same for the other categories, and for the other models. This way you can check if a model is predicting one category better than the other. For example, model 1 can be better at finding cars in a picture whilst model 3 can be better at finding buses.

The important part is that you know if the objects the model predicted actually correspond to what is in the picture. A very unlikely example would be a picture with 1 car and 1 truck where the algorithm recognizes the car as a truck and the truck as a car. From the info that is in the table I cannot be sure if the 2 cars you predict are the actual cars in the picture, or in other words, if they are actually True Positives or are actually False Positives.


Backpropogation rule for the output layer of a multi-layer network - What does the rule do in ambiguous cases? This is the back-propogation rule for the output layer of a multi-layer network:
$$W_{jk} := W_{jk} - C \dfrac{\delta E}{\delta W_{jk}}$$
What does this rule do in the more ambiguous cases such as:
(1) The output of a hidden node is near the middle of a sigmoid curve?
(2) The graph of error with respect to weight is near a maximum or minimum?

I assume you are considering a network where the activation function of the last layer is a sigmoid, so the output of your network is $$\tilde{y}=\sigma(W^{L}\cdot f(X, W^1, \dots, W^{L-1})),$$
where $X$ is the input vector, and $f$ is obtained by feeding the input to the network up to the layer $L-1$. Let's also call $Z:= W^{L}\cdot f(X, W^1, \dots, W^{L-1})$.
The error term is computed as $$E(y, \tilde{y})=E(y, \sigma(Z)),$$ where $y$ is the actual output. Let's get the derivative of the error with respect to the output of the last node (the input of the sigmoid)
$$\frac{\partial E}{z_i}=\frac{\partial E}{\partial \tilde{y}}\frac{\partial\tilde{y}}{\partial z_i}=\frac{\partial E}{\partial \tilde{y}}\frac{\partial\sigma}{\partial z_i}.$$
The update rule is $$z_i =z_i - C\frac{\partial E}{z_i}= z_i - C\frac{\partial E}{\partial \tilde{y}}\frac{\partial\sigma}{\partial z_i}.$$
Now we can analyse your questions.

To be close to the middle of the sigmoid means that $z_i$ is close to $0$; moreover the derivative $\frac{\partial\sigma}{\partial z_i}$ reaches its maximum value when it is evaluated in $0$. This means that the term $\frac{\partial\sigma}{\partial z_i}$ gets "large" as $z_i$ approaches the center of the sigmoid, contributing more to the update of the weight. Of course it is difficult to say what happens in general, as the term $\frac{\partial E}{\partial \tilde{y}}$ is also in the expression and it is possible that this term gets really small (or big) for $z_i$ close to $0$. You just know that the term $\frac{\partial\sigma}{\partial z_i}$ is trying to push $z_i$ further away from $0$, so the idea should be that the closer to the center the larger the update.
To be close to an extremum point of the loss means that the derivative of the loss with respect to $\tilde{y}$ is close to $0$. Since $\frac{\partial E}{\partial \tilde{y}}$ is backpropagated in a multiplicative fashion, the rule of thumb is that the closer you are to an extremum the smaller the updates get. Though, as above, is kind of difficult to say what will happen when updating a general node, as some of the term in the multiplication can be very large, making the update large even if $\frac{\partial E}{\partial \tilde{y}}$ is small.

For instance, what happens if you are close to the center of the sigmoid but also close to an extremum of the loss? You will have a multiplication of $2$ terms, one trying to make the update small and the other trying to make the update large, and what matter are the orders of magnitude involved.
In conclusion, the rules of thumb are as in the points 1. and 2., but they are no guarantee that you won't find any special cases.


Translating a single word Neural Networks I've been given an assignment to create a neural network that will suggest a Croatian word for a word given in any other European language (out of those found here). The words are limited to drinks you can find on a bar menu.

I've looked at many NN examples, both simple and complex, but I'm having trouble with understanding how to normalize the input.

For example, words "beer", "birra" and "cervexa" should all translate to "pivo". If I include those 3 in the training set, and after the network has finished training I input the word "bier", the output should be "pivo" again.

I'm not looking for a working solution to this problem, I just need a nudge in the right direction regarding normalization.

The training examples are not mentioned, but, assuming the assignment involves supervised learning, normalization is, in this case, mapping the inputs and outputs to binary values.

Translation, in this simplified context, is a map of strings to strings, which isn't really an AI device by contemporary standards. Since an artificial network is given as a requirement for the solution, the goal is to train the network to act like a hash map.

$$ \Big\{..., \;
\big(
[\text{"beer", "birra", "cervexa"}]:
\text{"pivo"}
\big)
, \; ...\Big\} $$

$$ \Downarrow $$

$$ \Big\{..., \; \big(\text{"beer" : "pivo"}), (\text{"birra" : "pivo"}), (\text{"cervexa" : "pivo"}\big), \; ...\Big\} $$

Assuming there is a list of pairs $(\mathcal{X}, \mathcal{Y})$, where $\mathcal{X}$ is the word in any of the source languages and $\mathcal{Y}$ is the word in the target language, and the vocabulary of both is known to have word counts of $X$ and $Y$ respectively, one can determine the input and output layer widths (cell counts) $w_1$ and $w_{\ell}$, where $\ell$ is the number of network layers.

$$w_1 = \text{ceiling} ( \log_2{X} )$$
 $$w_{\ell} = \text{ceiling} ( \log_2{Y} )$$

The assignment of source words to binary values is arbitrary. You can sort them by language and then by whatever common character encoding used (such as UTF-8) just for debugging purposes. The same is true of target words.

A simple MLP (multi-layer perceptron) with mini-batch training using basic propagation map be appropriate.


Loss/accuracy on Synthetic data I am trying to understand if there is any difference in the the interpretation of accuracy and loss on synthetic data vs real data.

No, there is no difference.

Of course, you are likely not able to extrapolate results obtained from synthetic data to expect identical or similar results in real life to unless you have very compelling reasons to do so.

Without a more specific question, I'm afraid a more specific answer is not possible.


Why would giving my AI more data make it perform worse? So I trained an AI to generate shakespeare, which it did somewhat well. I used this 10,000 character sample.

Next I tried to get it to generate limericks using these 100,000 limericks. It generated garbage output.

When I limited it to 10,000 characters, it then started giving reasonable limerick output.

How could this happen? I thought more data was always better.

The AI was a neural network with some LSTM layers, implemented in keras.

Why would giving my AI more data make it perform worse?

A lot of possible reasons:

In forecasting, you could have a seasonality. If you have it exactly 3 times, then it is good. If you have it 3.5 times, it becomes worse because it overfits on the months which occur more often than the others.
Data quality: In practice, improving the quality of data often yields better results (see Analysis and Optimization of Convolutional Neural Network Architectures, page 15 (Analysis Techniques) for 7 approaches to improve your results)
Instable training / bad luck: some architectures / problems are super instable. You can execute the randomly initialized training 5 times and get vastly different results.


Is there a way to run other platforms (other than Atari) in an OpenAI's Gym-like environment? Is there a way to run C64, Nintendo Gameboy, or other platforms in an OpenAI's Gym-like environment? It seems that we can run only Atari games.

Ok, I found

https://blog.openai.com/gym-retro/

but there may be other platforms?


What is the difference between search and learning? I came across an article, The Bitter Truth, via the Two Minute Papers YouTube Channel. Rich Sutton says...

One thing that should be learned from the bitter lesson is the great power of general purpose methods, of methods that continue to scale with increased computation even as the available computation becomes very great. The two methods that seem to scale arbitrarily in this way are search and learning.

What is the difference between search and learning here? My understanding is that learning is a form of search -- where we iteratively search for some representation of data that minimizes a loss function in the context of deep learning.

In the context of AI:

Search refers to Simon & Newell's General Problem Solver, and it's many (many) descendant algorithms. These algorithms take the form:

a. Represent a current state of some part of the world as a vertex in a graph.

b. Represent, connected to the current state by edges, all states of the world that could be reached from the current state by changing the world with a single action, and represent all subsequent states in the same manner.

c. Algorithmically find a sequence of actions that leads from a current state to some more desired goal state, by walking around on this graph.

An example of an application that uses search is Google Maps. Another is Google Flights.

Learning refers to any algorithm that refines a belief about the world through the exposure to experiences or to examples of others' experiences. Learning algorithms do not have a clear parent, as they were developed separately in many different subfields or disciplines. A reasonable taxonomy is the 5 tribes model. Some learning algorithms actually use search within themselves to figure out how to change their beliefs in response to new experiences!

An example of a learning algorithm used today is Q-learning, which is part of the more general family of reinforcement learning algorithms. Q-learning works like this:

a. The learning program (usually called the agent) is given a representation of the current state of the world, and a list of actions that it could choose to perform.

b. If the agent has not seen this state of the world before, it assigns a random number to the reward it expects to get for performing each action. It stores this number as $Q(s,a)$, its guess at the quality of performing action $a$ in-state $s$.

c. The agent looks at $Q(s,a)$ for each action it could perform. It picks the best action with some probability $\epsilon$ and otherwise acts randomly.

d. The action of the agent causes the world to change and may result in the agent receiving a reward from the environment. The agent makes a note of whether it got a reward (and how much the reward was), and what the new state of the world is like. It then adjusts its belief about the quality of performing the action it performed in the state it used to be in, so that its belief about the quality of that action is closer to the reality of the reward it got, and the quality of where it ended up.

e. The agent repeats steps b-d forever. Over time, its beliefs about the quality of different state/action pairs will converge to match reality more and more closely.

An example of an application that uses learning is AI.SEs recommendations, which are made by a program that likely analyzes the relationships between different combinations of words in pairs of posts, and the likelihood that someone will click on them. Every time someone clicks on them, it learns something about whether listing a post as related is a good idea or not. Facebook's feed is another everyday example.


What are the latest 'hot' research topics for deep learning and AI? I did my Master's thesis on Deep Generative Models and I'm currently looking for a new subject.

Q: What are the "hottest" research topics that are taking a lot of attention of the deep learning community lately?

A few clarifications:

I did look through similar questions and none of them answered my
question.
I come from a pure mathematical background, I only
transitioned into deep learning a year ago, and my research on
generative models was mostly theoretical. Which means, most of my
work revolved around structured probabilistic models, and approximate
inference. That said, I have yet to explore real world applications of deep learning.
I did my homework before posing the question. My goal was to get ai SE's input on the matter and see what people are working on.

The hot topics of today might be the cold, wet ashes of tomorrow.  For instance, the convergence speed of CNN and LSTM approaches, especially in combination, have diverted considerable attention away from basic RNN designs.

Similarly, the cold topics of today might be the burning embers of tomorrow. Of course, some of the cold topics will stay cold.  The sweet spot may be to identify those that are getting warmer and are likely to be sustainable building blocks future technology.

Residual Attention Networks

Residual attention networks, like LSTM networks, are an improvement over RNNs using a different approach.  Because attention networks are designed to conserve resources, they either converge faster or with less demand on hardware and network to support parallel execution.

Residual Attention Network for Image Classification, Fei Wang et al, 2017
Look Closer to See Better: Recurrent Attention Convolutional Neural
Network for Fine-grained Image Recognition, Jianlong Fu et all, 2017
VideoWhisper: Toward Discriminative Unsupervised Video Feature Learning With Attention-Based Recurrent Neural Networks, Na Zhao, 2017
Hierarchical Recurrent Attention Network for Response Generation
Chen Xing, Wei Wu et al, 2017

Automated Development of Non-Cartesian Models

Research into the automation of modelling is key to many AI applications.  Some of the algorithms under development do not simply extract tensors of features (arrays, matrices, cubes, and hyper-cubes), but develop graph models, directed or associative, with or without cycles permitted.

Hierarchical Self-organizing Maps System for Action Classification
Z Gharaee, P Gärdenfors, M Johnsson, ICAART, 2017
Runtime Modelling for User-Centric Smart Cyber-Physical-Human Applications, Lorena Castañeda Bueno, 2017
Probing the Topological Properties of Complex Networks Modeling Short Written Texts, Diego R. Amancio, 2015
Schema-summarization in Linked-Data-based feature selection for recommender systems, Azzurra Ragone et. al., 2017

Signal Topologies That Support Equilibria

Many ignore the importance of GANs, not because they can do interesting things with images but because of how they deviate from the simple topology of signal path where convergence on a trained set of parameters is achieved over a one-dimensional array of layers and blocks of layers.

The discriminative and generative components in GAN design are described in some detail in another AI Stack Exchange question on *Understanding the GAN Loss FUnction.  Although the generation of images from the GAN approach and its conceptual children demonstrate a new capability in the artificial network space, the breadth of this multi-network significance may not be immediately obvious.  It is not a stack in depth of layers, but a stack of two deep networks in a figure-eight topology, conceptually much like a Möbius strip.

This topology creates a balance between two networks, the generative (G) and the discriminative (D).  Its designer termed it an adversarial relationship because G and D play opposing roles.  However, their action in the system is actually collaborative, creating a balance that is much like a chemical equilibrium or symbiosis in biology, so that a specific objective is achieved.  This may reveal the most promising direction in AI today.

Designing signal topologies that support additional forms of collaboration and symbiosis between networks, where each network is a component that learns its roll in conjunction with other component networks, so that the aggregate system learns its function can synthesize forms of artificial intelligence that DNNs cannot.

Rules based systems and deep networks are one dimensional in terms of signal flow.  By themselves may never approximate the most notable features of the human brain.

Parallel Processing Using GPUs as DSPs

VLSI implementations of spiking networks is important, and there are now implementations such as https://github.com/Hananel-Hazan/bindsnet that leverage GPU hardware acceleration to investigate them without access to the VLSI chips being developed by large corporations.

Speech Recognition and Synthesis for End-to-End TTS

The recent emergence of excellence in synthesis using systems such as Google's WaveNet have opened the door to more accurate TTS (text to sound) applications, such that it is probably a good time to become an expert in voice recording for use in training example sets but a bad time to start a custom speech production house using live speakers.

Automated Vehicles

Automated vehicles of various types need experts in vehicle physics, automotive manufacture, aeronautics, and consumer products for a wide range of vehicle types with strong economic and safety incentives driving semi-automation and full automation.

Mars landers
Consumer drones
Industrial drones
Military drones
Passenger aircraft
Passenger automobiles
Limos
Trains
Wheel chairs
Delivery vehicles
Automated food distribution
Nuclear plant repair robots
Electrical distribution repair robots

Summary

It may be difficult to discover in advance what of hot technologies in AI will remain dominant in five years or which of the warming technologies will be blazing hot then, but the above are solid technologies showing significant early promise and for which there are high business, industrial, and consumer demands.


How GoogleNet actually deal with reducing overfitting? Today I was going through a tutorial of Andrew Ng about Inception network. He said that GoogLeNet's hidden layers are also good in prediction and it had somehow a regularization effect, so it reduces overfitting. I also search on this topic and tried to figure out by reading the GoogLeNet paper. But I am not satisfied.

Can anyone give me any mathematical intuition or reasoning about this in detail?

The main reason of overfitting in any neural network is having too many unrestricted trainable degrees of freedom in the model. Methods similar to dropout reduce the number of neurons at each training run which effectively means having a smaller network. On the other hand in $l_1$  and $l_2$ regularization, a term added to the loss function which put a constraint on the total loss calculated at each run. So what we are trying to minimize with such regularizations is not just $L$, but $L +l_1*f(w)$ (for example).

What I understand from that paper is that the auxiliary outputs do both at the same time: the point is to build smaller version of the same inception network within itself, and use the losses obtained from those as a constraint on the final loss function. The full model described in the paper is essentially 3 separate models: the first one is a network with 3 inception modules, the second one with 6 and the final one has 9. When loss is calculated, results from the auxiliary outputs added to the total loss with a 0.3 weight. Let us write this as follows:

$L = 0.7*L_9+ 0.3*(L_6+L_3).  $

Here $L_3$ and $L_6$ represent the losses calculated at first and second outputs, and $L_9$ is the loss calculated at the final output of the network.

This is the function we wish to minimize during training. But when the evaluation is made, the auxiliary outputs are discarded, and just the final softmax layer is used. Not very dissimilar from the idea of using dropout during training but using the full model for predictions.


Is my understanding of how AI works correct? In my discussion over my question on Math SE, I explained to a user, how I think AI works, I wrote that with the sigmoid(logistic) function, features of a data set are identified, many such iterations provide learning.

Is my understanding of how this works correct?

There's some useful information in your description, but that's just a very vague description of how neural networks with sigmoid activation functions are trained.

Moreover, there are many other AI systems apart from neural networks (such as support vector machines, expert systems, etc.), which, of course, I cannot exhaustively list here.

  Is my understanding of how AI works correct?

I would say it's not completely incorrect, but, as I said, it's a very vague description and it only refers to a subset of techniques in the AI field. With that description, no newbie would probably understand how neural networks are really trained, apart from knowing that you will train them iteratively and there are sigmoids involved.


Where to publish a first article in Deep Reinforcement Learning? What would be examples of journals that are good for a first publication in the field of Deep Reinforcement Learning?
I am in the process of writing about the research results of DQN-related algorithms.
I have 3 requirements - it should be indexed in one of these databases, otherwise, I cannot receive grant money for research:

https://www.scopus.com/
http://webofknowledge.com

And it should not be very expensive to publish. It should be under 1000EUR to publish, for example, the Open Access license for Elsevier "Artificial Intelligence" journal costs around 2400EUR to publish.
And it should not have a very long review/publishing period. For example, Elsevier's "Information Fusion" journal currently gathers articles for July 2018, which is 8 month period till publishing. Is it normal?
Can you please recommend some journals that qualify & you have had good experience publishing research?

I recommend you focus on quality over quantity. Publishing a paper will boost your reputation and make you more recognised within your academic field (AI); however, this is only if the paper provides useful insights into an important issue.
Your paper is more likely to be accepted if it is well written and easy to understand, stimulates new important questions, uses rigorous methods to explain why the data supports the conclusion and connections to prior work is made and serve to make your paper's arguments clear. (Elizabeth Z Elsevier blog)
Before submitting your paper, ask a mentor or a colleague to proofread it, so that you can make the relevant revisions and changes. Journal editors will look down on your work if it is poorly written or contains substandard grammar.
A way to get published is by writing reviews, especially for researchers in earlier stages of their careers. Most journal editors like to publish replies to previous publications since it stimulates debate.
Remember it is acceptable to challenge reviewers' suggestions with good justification. Many researchers fail to persevere when they are instructed to revise and resubmit their work. Don't give up, however, you can politely decline or even argue why a reviewer is wrong. Editors will accept a rational explanation if it is clear that you have considered all their feedback.
Getting published is never easy, especially in high ranking journals. If you focus on getting published quickly it could derail you from concentrating on the quality of your research. Yes, getting published can be expensive, however, it's much better for your career if you write a high-quality paper than a low-quality paper in a lowly ranked or ungraded journal since it will not be REFable.
Below is a list of Artificial Intelligence Journals that you can submit your papers to and possibly get published.

Artificial Intelligence (Journal) https://www.journals.elsevier.com/artificial-intelligence

Artificial Intelligence Review  https://link.springer.com/journal/10462

Applied Artificial Intelligence http://www.tandfonline.com/action/journalInformation?journalCode=uaai20&

Autonomous Agents and Multi-Agent Systems  https://www.springer.com/computer/ai/journal/10458

International Journal on Artificial Intelligence Tools http://www.worldscientific.com/worldscinet/ijait

International Journal of Pattern Recognition and Artificial Intelligence http://www.worldscientific.com/worldscinet/ijprai

Journal of Experimental and Theoretical Artificial Intelligence  http://www.tandfonline.com/action/journalInformation?journalCode=teta20

Journal of Machine Learning Research  http://www.jmlr.org/

IEEE Intelligent Systems  http://publications.computer.org/intelligent-systems/

Journal of Automated Reasoning  https://www.springer.com/computer/theoretical+computer+science/journal/10817

Minds and Machines  https://www.springer.com/computer/ai/journal/11023

Autonomous Agents and Multi-Agent Systems  https://www.springer.com/computer/ai/journal/10458

Applied Intelligence  https://link.springer.com/journal/10489

Journal of Intelligent Manufacturing  https://link.springer.com/journal/10845

Journal of Intelligent Information Systems   https://link.springer.com/journal/10844

AI & Society  https://link.springer.com/journal/146


Can I reduce the "number of weights" in CNN to 1/3 by restricting the input as greyscale image? In a CNN, does each new filter have different weights for each input channel, or are the same weights of each filter used across input channels?

This question helps me a lot.

Let, I have RGB input image. (3 channels)
Then each filter has n×n weights for one channel.
It means, actually the filter has totally 3×n×n weights.

For channel R, it has own n×n filter.

For channel G, it has own n×n filter.

For channel B, it has own n×n filter.

After inner product, add them all to make one feature map.
Am I right?

And then, my question starts here.
For some purpose, I will only use greyscale images as input.
So the input images always have the same values for each RGB channel.

Then, can I reduce the number of weights in the filters?
Because in this case, using three different n×n filters and adding them is same with using one n×n filter that is the summation of three filters.

Does this logic hold on a trained network?
I have a trained network for RGB image input, but it is too heavy to run in real time.
But I only use the greyscale images as input, so it seems I can make the network less heavy (theoretically, almost 1/3 of original).

I'm quite new in this field, so detailed explanations will be really appreciated.
Thank you.

After inner product, add them all to make one feature map. Am I right?

yes, you are right.

  Then, can I reduce the number of weights in the filters? Because in this case, using three different n×n filters and adding them is same with using one n×n filter that is the summation of three filters.

If you have transformed the image into greyscale then you no longer need 3 filters. You should retrain your model on greyscale images. In a greyscale image the value of each pixel is a single sample representing only an amount of light (the light intensity).

The network will run faster if that is the only architectural change you make, but keep in mind that by converting the image to greyscale you will lose information and probably some of the predictive power of your network.


Is there such a thing like the machine learning paradox? O'Reilly recently published an article about the machine learning paradox. (link)

What it says goes basically like this: no machine learning algorithm can be perfect. If it was, it means it is overfitting and so it is not really perfect because it will not perform ok in real world scenarios.

I searched and I couldn't find any other references to this paradox. The closest I got is the Accuracy Paradox, which says that the usefulness of a model is not really well reflected in its accuracy.

This doesn't sound quite ok to me. For example, a linear model could be perfectly learned, "overfitted" and predicted in the real world. So I suspect it is really about finding the right set of data points from which the results can be inferred. This is, we are trying to approximate from uncertain data, but with the right data we can stop approximating and start calculating.

Is my line of thinking correct? Or is there really no perfect machine learning?

UPDATE: In the light of currently received answers, I think my last paragraph (my line of thinking) can be rephrased as: If we have a model simple enough, why can't we overfit the model, knowing that it will behave correctly in non-trained data? This assumes the training data completely represents the real-world data, which would imply a single model that we can train on.

Keep in mind that what we conceive as "simple" or "feasible" is arbitrary and only depends on computation power and available data -- aspects with are external to ML models themselves.

Basically Machine Learning is used to solve problems for which giving exact solutions is not (practically or theoretically) possible or feasible. If you have a problem that can be solved with exact and proven methods, you don't need Machine Learning. So Machine Learning is rather about finding solutions by approximations with some error margins. In this sense such systems will never perform perfectly and there always will be room for improvements.
(Many real world problems can not be solved perfectly because they are not simply complicated, but chaotic.)


Iteratively and adaptively increasing the network size during training For an experiment that I'm working on, I want to train a deep network in a special way. I want to initialize and train a small network first, then, in a specific way, I want to increase network depth leading to a bigger network which is subsequently to be trained. This process will be repeated until one reaches the desired depth. It would be great if anybody heard of anything similar and could point out to me some related work. I think in some paper I read something about a related technique where people used something similar but I don't find it anymore.

Neuroevolution Through Augmenting Topologies or NEAT may be what you are referring to. The original paper by Kenneth O. Stanley is here

NEAT combines a neural network and a genetic algorithm. Instead of using back propagation or gradient descent to "train" your network, NEAT creates a population of very simple neural networks (no connections) and evolves them with fitness evaluation, crossover, and mutation. The genome syntax: every connection gene has a few settings. In node, Out node, Weight of connection, activated, and innovation. In, Out, and Weight values are the same as regular neural networks. Enabled and Disabled genes are well, enabled and disabled. The innovation value is possibly the most defining feature of NEAT, since it allows for crossover of different topologies and historical tracking of each connection.NEAT can mutate or change both its weights and connections, so for example, Parent1 and Parent2 has 5 of the same connections, represented by innovation / ID numbers 1 through 5. Since they have the same connection nodes, the genetic algorithm will randomly pick either Parent1 weight or Parent2 weight. The excess and disjoint genes are inherited from the more fit parent. NEAT will then mutate each genome, shown in the image below.


State of the art and possible development of judging AIs in laws field I'd like a general explanation of that in AIs that were to mimic judges, prosecutors or lawyers, on very general terms they would act on this way for each case:

A judge AI would give a verdict, having the following input:

All sources of law that have some relationship with the act being judged, knowing the relative importance of each of that sources.

All the facts presented to it, being theorically able to distinguish if some fact were false if there are enough evidence for that.

Prosecutors AI would act into searching the facts that would most help for accusing, matching them with known sources of law so they can make the best accusation.

Lawyers AI would act exactly as prosecutors but for defense of the one being accused.

Obviously AIs are way too far nowadays from something like this, but I find strange that it doesn't even look that this has been done as a proof of concept, the most near thing I find by searching on the Internet is this. I'm not sure of the reach of everything that it's explained there but it looks more of assisting judges, prosecutors and lawyers rather than making what they do. I find this a bit strange, although I'm very far from being an expert on AIs, I think that for very simple cases an AI could even do those things.

In the answer about possible development, I'd like to know about which would be the facts that would make developing an AI like the one I'm mentioning to be possible or not.

PD: I don't find any suitable tag, I ask an admin to put a better one if needed.

Requested edit: My question is about the general state of the art of this field (of AIs being able to somehow mimic a judge, a prosecutor or a lawyer), and how pausible is a development in this way and how it would develop (if this is with current knowledge impossible to even try until a far future, or if we are at least near for stablishing AIs understanding law, which would be the logical way for a type of AIs like them evolving (understing laws, judging, being able to see videos..). I know it's broad so I'm happy with a general explanation.

There has definitely been work relating to judicial sentencing:

  In a new study from the National Bureau of Economic Research, economists and computer scientists trained an algorithm to predict whether defendants were a flight risk from their rap sheet and court records using data from hundreds of thousands of cases in New York City. When tested on over a hundred thousand more cases that it hadn’t seen before, the algorithm proved better at predicting what defendants will do after release than judges.Source: M.I.T. Technology Review: How to Upgrade Judges with Machine Learning

In terms of actually trying cases to deliver a verdict of guilty or not guilty, I think that would be difficult because, although Machine Learning has been effective at outperforming humans in problems where parameters can be clearly defined, we haven't figured out how to tackle the symbol grounding problem. This means that making decisions based on language, which is less precise than mathematics, would be highly problematic. Guilty/Not Guilty is a basic True/False condition, but the rest of the language utilized in law cannot be so precisely translated.


Issue with simple game AI A few months ago I made a simple game that is similar to the dinosaur game in Google Chrome - you jump over obstacles, or don't jump over levitating obstacles, and jump to collect bitcoins, which can be placed at 5 different heights. I used a very lightweight NN written by NYU professor Dan Shiffman, and within a few days the game and AI were done, starting off with a population of 200 jumpers, and a genetic algorithm (fitness function (points are given for avoiding obstacles and gathering bitcoins) and mutation), and it worked as it should.

However, this was only when the bitcoins and obstacles were not near each other, which I've been struggling with ever since.

So, I made a "training ground" where I put first a levitating obstacle, then a grounded one, and then a bitcoin after it, and then a bitcoin above a fourth grounded obstacle, and no matter how many times and how long I'd leave it to train, I'd always end up with identical behavior:

The first 3 obstacles are properly avoided, the first bitcoin is collected, and then jumpers would jump too early, land before the fourth "bitcoin" obstacle, and jump again, always crashing at almost the same place (across all generations, so even if I'd restart the training, they crash at the same place in the obstacle, with a deviation of a few pixels up or down).
I added multilayer support to the NN, no improvements.

Today I replaced the NN with tensorflow.js, and I am getting identical behaviour.

My inputs are:

distance to next obstacle
altitude of next obstacle
distance to next star

(for simplicity I removed the altitude of stars from the input, and keep them at a constant altitude)

I have 2 hidden layers (5 and 6 neurons), and 1 neuron in the output, which determines if the jumper should jump.

My only idea is that a neuron that decides when to jump because of the obstacle activates alongside the neuron that decides when to jump because of the bitcoin, their weights are summed up and a decision to jump too early is made.

I'll give somewhat of a (maybe bad) analogy:

If it takes you 1 month to prepare an exam, then, if you have 2 exams on the same day, you will start preparing them 2 months earlier. That logic works in this case, but not in my AI.

In the initial "toy neural network" I even added 8 layers of 12 neurons each, which I think is overkill for this case.
In tf.js I used both sigmoid and relu activation functions.
No matter what I did, no improvement.

Hope someone has an idea where I'm going wrong.

The issue is likely in how you estimate wellness, how the error function is constructed and from what data, since you have used two known good pieces of software and probably known good derivatives for your activation functions.  The second most likely is in now the components of wellness is aggregated.  Summing squares is sometimes not representative of a solid aggregation strategy.

I'm a bit confused about the game for three reasons.

Stars enter the description without telling us whether they are not obstacles, are obsticles, or the only obstacles
At one point bitcoins collection is the objective and obstacles are the challenge and at another point bitcoins are obstacles themselves
Your inputs have distance but not direction (Is this 2-D game?)

Based on textual hints, I'm going to assume five simple things, and you can correct any misconceptions I list.

Collecting all bitcoins is part of the objective
Running into bitcoins mid-jump is considered a crash
There is some radial tolerance to landing in at a bitcoin location with some imprecision
There are other things to crash into
Your outputs are jump magnitude and direction

I see that when the bitcoins are not in the vicinity of other things, the net can be trained effectively to jump to them, but when in the vicinity of other things, the training converges on a behavior that repeatedly fails prior to completion.  I'm assuming that the failure location is not 100% repeatable because the genetic algorithm has a pseudo-random seed that changes.  Again, correct me if I misunderstand in my piecing together the scenario.

One should consider the possibility that the distance to non-bitcoin obstacles is part of the error function and the difference between the jump destination and the bitcoin is also part of the error function.  (This second one is why I prefer to call the error contour a wellness measure.)

If the bitcoin incentive does not seem to be the crash cause, in that, had the other obstacle been moved, the bitcoin would have been collected, then the first of the two wellness criteria needs a higher order contribution from the distance to the other obstacle.

There are two simple functional forms that come to mind, which could be tried that increase the alarm represented in back propagation when the probability of collision is heightened to more effectively train against collision.  Both involve determining the direct jump line to the nearest bit coin and the distance from that line to its nearest other obstacle; call that x.

$x^y$, where $y > 1.0$
$e^{kx}$


When using experience replay in reinforcement learning, which state is used for training? I'm slightly confused about the experience replay process. I understand why we use batch processing in reinforcement learning, and from my understanding, a batch of states is input into the neural network model.
Suppose there are 2 valid moves in the action space (UP or DOWN)
Suppose the batch size is 5, and the 5 states are this:
$$[s_1, s_2, s_3, s_4, s_5]$$
We put this batch into the neural network model and output Q values. Then we put $[s_1', s_2', s_3', s_4', s_5']$ into a target network.
What I'm confused about is this:
Each state in $[s_1, s_2, s_3, s_4, s_5]$ is different.
Are we computing Q values for UP and DOWN for ALL 5 states after they go through the neural network?
For example, $$[Q_{s_1}(\text{UP}), Q_{s_1}(\text{DOWN})],
\\ [Q_{s_2} (\text{UP}), Q_{s_2}(\text{DOWN})], \\
[Q_{s_3}(\text{UP}), Q_{s_3}(\text{DOWN})], \\
[Q_{s_4}(\text{UP}), Q_{s_4}(\text{DOWN})], \\
[Q_{s_5}(\text{UP}), Q_{s_5}(\text{DOWN})]$$

The way the states are used is as follows:
Typically your $Q$-network will state a state as input and output scores over the action space. I.e. $Q : \mathcal{S} \rightarrow \mathbb{R}^{|\mathcal{A}|}$. So, in your replay buffer you should store $s_t, a_t, r_{t+1}, s_{t+1}, \mbox{done}$ (note that done just represents where the episode ended on this transition and I add for completeness.
Now, when you are doing your batch updates you sample uniformly at random from this replay buffer. This means you get $B$ tuples of $s_t, a_t, r_{t+1}, s_{t+1}, \mbox{done}$. Now, I will assume $B=1$ as it is easier to explain and the extension to $B > 1$ should be easy to see.
For our state-action tuple $s_t, a_t$ we want to shift what the network predicts for this pair to be closer to $r_{t+1} + \gamma \arg\max_a Q(s,a)$. However, our neural network only takes the state as input, and outputs a vector of scores for each action. That means we want to shift the output of our network for the state $s_t$ towards the target I just mentioned, but only for the action $a_t$ that we took. To do this we just calculate the target, i.e. we calculate $r_{t+1} + \gamma \arg\max_a Q(s,a)$, and then we do gradient ascent like we would a normal neural network where the target vector is the same as the predicted vector everywhere except the $a_t$th element, which we will change to $r_{t+1} + \gamma \arg\max_a Q(s,a)$. This way, our network moves closer to our Q-learning update for only the action we want, in line with how Q-learning works.
It is also worth nothing that you can parameterise your Neural Network to be a function $Q: \mathcal{S} \times \mathcal{A} \rightarrow \mathbb{R}$ which would make training more in line with how tabular Q-learning but is seldom used in practice as it becomes much more expensive to compute (you have to do a forward pass for each action, rather than one forward pass per state).


What is a heavy node in neural networks? I was watching a documentary on Netflix about AlphaGo, and at one point (~1:10:16 from the end), one of the programmers uses the term "heavy node," which I assume has to do with neural networks. I did a little bit of research but couldn't find anything on what that term means. The closest I could get was this wikipedia page on Heavy path decomposition: https://en.wikipedia.org/wiki/Heavy_path_decomposition, which seemed like it could be somewhat related, but I wasn't sure how exactly. Has anyone heard of this term being used? Does anyone know what it means?

For context, in the documentary the line is that if it (the network/player) creates something new not in the heavy node, then they don't see it.

Yes, firstly you are correct to acknowledge the term node with reference to the neural network.

A neural network is made to mimic the working of a human brain hence the term neural. It comprises of several layers and each layer comprises of several nodes. A network can be a deep neural network, signifying the presence of numerous layers, in which the output of the one, serves as an input for the other.
Further a node in a particular layer has some weights and biases attached with it that are responsible for the computation and producing output.

To your context, a heavy node may refer to a node that holds importance for the network and has got significant weight and bias attached. Henceforth as per the statement if the network does make any significant change to the node with importance in terms of weights and biases, it is noticed else not.


Is there an alternative to the use of target network? In the context of Deep Q Network, a target network is usually utilized. The target network is a slow changing network with a changing rate as its hyperparameter. This includes both replacement update every $N$ iterations and slowly update every iteration.

Since the rate is hard to fine tune manually, is there an alternative technique that can eliminate the use of target network or at least makes it less susceptible to the changing rate?

I have done some research and would like to share.

Generally to eliminate the use of target network one needs to show that training would be stable under off-policy semi-gradient.

There are two approaches that might work:

Experience reweighting
Constrained optimization

Experience reweighting

Probably the simplest idea is to use importance sampling ratio (Precup 2001) to multiply each sample. This would correct the off-policy sample distribution to be on-policy distribution. It has been shown (Sutton 2016) that on-policy samples lead to stability for semi-gradient. However, this line of work that corrects for sample distribution has high variance and does not work well in practice.

Another line of work aimed to partially correct the distribution only to the extent that would be provably stable is called Emphatic TD (Sutton 2016). The distribution is still mostly off-policy but is proved to be stable under linear function approximation.

Constrained optimization

The general wisdom comes from the fact that updating a value would also alter the target value since they share the same set of parameters. This problem is called over-generalization. To reduce this, Durugkar suggests that constraining the target Q-value after parameter update to be steady helps reduce divergence. Achiam 2019 provides a good overview of the problem, and suggests a way to make sure that the update is non-expansion (which should prevent divergence).

Sidenote

Some works have shown convergence under off-policy samples but only in tabular case. It takes much more to show that it is stable in function approximation (even a linear one).

Works like:

Q($\lambda$) (Harutyunyan 2016), which augments the value function with a correction term
Retrace($\lambda$) (Munos 2016), which extends the former by truncating the importance sampling ratio

are shown to work only in tabular case.


How can I generate unique random patterns (similar to the ones in Nutella jars)? How can I generate unique patterns, as they did for these Nutella jars? See, for example, the video Algorithm designs seven million different jars of Nutella.

Some excerpts from     Nutella 'Hired' an Algorithm to Design New Jars. And It Was a Sell-Out Success:

The "algorithm" is called HP Mosaic and is included free in HP SmartStream Designer for HP printers.

More about how the algorithm works here: https://www.linkedin.com/pulse/hp-mosaic-20-steven-chow

HP Mosaic takes the vector PDF file as input (also known as a Seed file), and generates a large number of variations on the file by transforming it — scaling, transposition, and rotation — randomly.


Evaluate precision and recall results The following table shows the precision and recall values I obtained for three object detection models.

I evaluate the first two models as the following. The target is to find the best object detection model for that particular data set.

  Model 1 has a high recall and precision values. High precision relates to a low false-positive rate, and high
  recall relates to a low false-negative rate. High scores for both show
  that the model is returning accurate results. Model 2 has high
  precision but low recall. This means it returns very few results, but
  most of its identified objects are correct.

How can I evaluate the third one?

This looks like it could be a homework problem, so consider updating it with the homework tag if so.

The second model has the same precision, but worse recall, than model 1. Therefore we would rather have model 1 than model 2.

The third model has worse recall than model 1, and worse precision than model 1, therefore we would rather have model 1 than model 3.

Thus, model 1 is the best object detection model.


How to correctly label images for multi-label classification? I have images that contain lots of elements. Some I know, some I don't. I want to know if it's ok to only label those I do know. Let's take this image for example. I would label the green stuff and the worm but leave the rest unlabeled.
Is that ok?

Another question I would also like to ask is how concise I should be in labeling. For instance, You can see in the picture a bit of blue behind the green plant. So should I label that bit and say water or leave it unlabeled?

EDIT:

I also want to ask if it's ok to label only the things I'm interested in even if they take up to  30% of the picture? Won't the neural network be confused by all the details in the picture that it perceives and that I label as A for example, even if A is just a part of it?

Another question would be, let's say I have labels A, B and C. I have an image in which I'm a bit confused if a certain object is of label B or A or even a totally different class other than (A,B,C). What should I do in this instance?

I'm having a hard time with the dataset. It would take an expert to label this correctly. But I want to do things as cleanly as possible, so all the effort doesn't go to waste. I would really appreciate your help. Thank you guys.

You can't label things you don't know.  The goal of labeling is to label the things you want the classifier to learn so that when you run it in inference mode you can discover what is in your data (new data that you didn't use for training, validating, or testing).

It is not a good idea to label small objects like the 'blue water' unless it is important to you to discover these fine details in inference mode.


Image recognition service architecture A question for developers of projects for pattern recognition. How best to organize the architecture of such a service?

At what stage do you conduct logic? (for example, for the recognition of a photo of a male blue jacket, a cascade of queries is performed: "recognizing men" -> "recognizing the jacket" -> "recognizing the color of the jacket.")

Does it make sense to implement all search options within a single neural network or is it better to create a set of individual neuronets that are confined to fairly simple tasks?

That is one of the good example for research. Personally, I prefer to segment out all the desired outputs at once. Then, check the success rate. If you cannot hit the success rate that you desire, you can go for more specific solutions for the specific problem that you face.

However, in general, the localization, segmentation, recognition are implemented in same network and are obtained all-at-once.


Can we train the model to detect real users with only positive labels? We have hundreds of thousands of customers records, and we need to take the benefits of our data to train a model that will recognize fake entries or unrealistic ones for our platform, where customers are asked to enter their names, phone number and zip code.

So, our attributes are name, phone number, zip code and IP address to train the model with. We have only data associated with real users. Can we train a model provided with only positive labels (as we do not have a negative dataset to train the model with)?

The problem which you have is a classification problem. You assume a class "good users" and a distinct class "bad users". You want to train an AI to tell the two apart, but all your examples are "good users". Any reasonable AI will draw the logical conclusion from those examples: all users are good users. That's a 100% match for the training data.


How to properly optimize shared network between actor and critic? I'm building an actor-critic reinforcment learning algorithm to solve environments. I want to use a single encoder to find representation of my environment.

When I share the encoder with the actor and the critic, my network isn't learning anything:

class Encoder(nn.Module):
  def __init__(self, state_dim):
    super(Encoder, self).__init__()

    self.l1 = nn.Linear(state_dim, 512)

  def forward(self, state):
    a = F.relu(self.l1(state))
    return a

class Actor(nn.Module):
  def __init__(self, state_dim, action_dim, max_action):
    super(Actor, self).__init__()

    self.l1 = nn.Linear(state_dim, 128)
    self.l3 = nn.Linear(128, action_dim)

    self.max_action = max_action

  def forward(self, state):
    a = F.relu(self.l1(state))
    # a = F.relu(self.l2(a))
    a = torch.tanh(self.l3(a)) * self.max_action
    return a

class Critic(nn.Module):
  def __init__(self, state_dim, action_dim):
    super(Critic, self).__init__()

    self.l1 = nn.Linear(state_dim + action_dim, 128)
    self.l3 = nn.Linear(128, 1)

  def forward(self, state, action):
    state_action = torch.cat([state, action], 1)

    q = F.relu(self.l1(state_action))
    # q = F.relu(self.l2(q))
    q = self.l3(q)
    return q

However, when I use different encoder for the actor and different for the critic, it learn properly.

class Actor(nn.Module):
def __init__(self, state_dim, action_dim, max_action):
    super(Actor, self).__init__()

    self.l1 = nn.Linear(state_dim, 400)
    self.l2 = nn.Linear(400, 300)
    self.l3 = nn.Linear(300, action_dim)

    self.max_action = max_action

def forward(self, state):
    a = F.relu(self.l1(state))
    a = F.relu(self.l2(a))
    a = torch.tanh(self.l3(a)) * self.max_action
    return a

class Critic(nn.Module):
  def __init__(self, state_dim, action_dim):
    super(Critic, self).__init__()

    self.l1 = nn.Linear(state_dim + action_dim, 400)
    self.l2 = nn.Linear(400, 300)
    self.l3 = nn.Linear(300, 1)

  def forward(self, state, action):
    state_action = torch.cat([state, action], 1)

    q = F.relu(self.l1(state_action))
    q = F.relu(self.l2(q))
    q = self.l3(q)
    return q

Im pretty sure its becuase of the optimizer. In the shared encoder code, I define it as foolow:

self.actor_optimizer = optim.Adam(list(self.actor.parameters())+
                                      list(self.encoder.parameters()))
self.critic_optimizer = optim.Adam(list(self.critic.parameters()))
                                         +list(self.encoder.parameters()))

In the seperate encoder, its just:

self.actor_optimizer = optim.Adam((self.actor.parameters()))
self.critic_optimizer = optim.Adam((self.critic.parameters()))

two optimizers must be becuase of the actor critic algorithm, in which the loss of the actor is the value.

How can I combine two optimizers to optimize correctly the encoder?

Just use one class inheriting from nn.Module called e.g. ActorCriticModel.

Then, have two members called self.actor and self.critic and define them to have the desired architecture.Then, in the forward() method return two values, one for the actor output (which is a vector) and one for the critic value (which is a scalar).

This way you can use only one optimizer.


Embedding Isolation game states into key values for RL I'm trying to think of how I can embed a game's state into a unique key value. The game I'm specifically working with is Isolation: https://en.wikipedia.org/wiki/Isolation_(board_game). The game state has the coordinates of player 1's pawn, coordinates of player 2's pawn, coordinates of free spaces and coordinates of already used spaces. Is there a way to embed this into a unique key value? My plan is to generate a dict and use that for value iteration with RL to learn the optimal value function for every state.

I think that there are too many game states in that game for you to use value iteration. The upper bound for a simple concise representation would be $49^2 \times 2^{49}$. That is:

$49^2$ covering all possible locations of the two players.

$2^{49}$ covering whether each square exists or has been removed.

In this scheme, many combinations are not feasible, as pieces do have to be on existing squares and cannot share a space. However, this doesn't drop the number of allowed states by a significant amount (several or more orders of magnitude) that would make it worth a more complex representation, or put the game into reach of dynamic programming solutions.
Potentially, a version on a 4x5 grid would be small enough to fit in memory and be solvable with dynamic programming. That would have a few million valid states.
In terms of an id code, you could use a 64-bit unsigned integer, reserving 49 bits for the existence of the squares, and for simplicity giving 6 bits each to the locations. This would also be a valid state representation for actually playing the game efficiently, most programming languages support the bit manipulations that you would need in order to maintain the representation. However, it would need a separate expanded representation if you were to create neural network features for approximate value functions.
In terms of writing a solver for the 7x7 version, I would recommend combining a look-ahead planner such as negamax, with an approximate neural network-based reinforcement method, perhaps DQN for simplicity. The neural network would provide a backup "best guess" solution when you could not look far enough ahead in the early stages of the game.
On a quick search for the true number of states in this game, I found an introduction to using minimax on a smaller version that you may find useful.


Speeding up CNN training So I built a CNN without any scientific libraries like TensorFlow or Keras (only NumPy). It is taking a huge amount of time to train. What are some of the tricks and tips followed by people to speed up training of a CNN? (I am not talking about division of jobs into different processors but subtle redundant codes i.e. giving pre-calculated results which is not visible to common programmers).

[Ref-some standard checks performed by programmers]

Speeding up Convolutional Neural Networks with Low Rank Expansions

From the abstract:

  The focus of this paper is speeding up the evaluation of convolutional neural networks. While delivering impressive results across a range of computer vision and machine learning tasks, these networks are computationally demanding, limiting their deployability.

  Convolutional layers generally consume the bulk of the processing time, and so in this work we present two simple schemes for drastically speeding up these layers. This is achieved by exploiting cross-channel or filter redundancy to construct a low rank basis of filters that are rank-1 in the spatial domain.

  Our methods are architecture agnostic, and can be easily applied to existing CPU and GPU convolutional frameworks for tuneable speedup performance. We demonstrate this with a real world network designed for scene text character recognition, showing a possible 2.5x speedup with no loss in accuracy, and 4.5x speedup with less than 1% drop in accuracy, still achieving state-of-the-art on standard benchmarks.


What is the difference between Sutton's and Levine's REINFORCE algorithm? I followed the videos/slides of Berkley RL course, but now I am a bit confused when implementing it. Please see the picture below.

In particular, what does $i$ represent in the REINFORCE algorithm? If $\tau^i$ is the trajectory for the whole episode $i$, then why don't we average across the episodes $\frac{1}{N}$, which approximates the gradient of the objective function? Instead, it is a sum over the $i$. So, do we update the gradients per episode or have batches of episodes to update it?
When I compare the algorithm to Sutton's book as shown below, I see that there we update the gradients per episode.

But wouldn't it then contradict the derivation on the Levine's slide that the gradient of the objective function $J$ is the expectation (therefore sampling) of the gradients of the logs?

Secondly, why do we have a cumulative sum of the returns over $T$ in Sutton's version but do not do it in Levine's (instead, all returns are summed together)

About the first question, you are right. The $i$ denotes a sample trajectory corresponding to a whole episode. However, Sutton's version is exactly the same one as Levine's if you choose $N=1$.

About the second question, the Policy Gradient theorem only tells you what is the gradient up to a constant, so basically any constant is irrelevant. Now, even if you do know the constant, you are going to multiply the gradients by an arbitrary learning rate $\alpha$. So, you can think that the factor $\frac{1}{N}$ is actually already considered "inside" $\alpha$.


Which one is more important in case of different loss optimization algorithms, Speed or the Route? We have different kinds of algorithms to optimize the loss like AdaGrad, SGD + Momentum, etc. Some are more commonly used than the others. In some algorithms, they usually range out before they converge, reach to the steepest slope and find the minima. But some of these algorithms are significantly fast. So my question is that the speed is more of a deciding factor here or the route is important too? Or is it just problem dependent?

Here is a picture of what I mean by the Route.

The route/trajectory followed by the optimization algorithm basically depends your dataset and the loss function. However, what really matters, for the purpose of final accuracy performance, is the final point which the trajectory converges to.


Is it mostly the case to train with available models I quite often find projects using pre-trained model and using them as a starting point for their new model that learns something novel from thier dataset or on-live learning process - e.g. using a webcam or live audio.

Is this quite usual and recommended to speed up training a model? For example using a model trained on ImageNet as a first layer to your model that will categorise faces specifically.

Yes, it is recommended to start with pre-trained model if you don't have high-end hardware. You can use a pre-trained model for fine-tuning (their trained weight as your initial weight) or use it as feature extractor (you remove few last layers, and then train it).
Why we need a pre-trained network?

Because training a good deep model takes a lot of time and needs a lot of hardware. Some good models, like DenseNet, ResNet, even VGG16 need days of training. You can read from VGG original paper:

On a system equipped with four NVIDIA Titan Black GPUs, training a single net took 2–3 weeks depending on the architecture.

Sometimes our case is similar to the dataset of pre-trained models out there. For example, if we need to classify some images of flower type, even our case has different class, we can use some first Convolutional layers of pre-trained model and use it as a feature extractor.

You can read more about transfer learning from this paper or this page.


Generation of 'new log probabilities' in continuous action space PPO I have a conceptual question for you all that hopefully I can convey clearly. I am building an RL agent in Keras using continuous PPO to control a laser attached to a pan/tilt turret for target tracking. My question is how the new policy gets updated. My current implementation is as follows

Make observation (distance from laser to target in pan and tilt)
Pass observation to actor network which outputs a mean (std for now is fixed)
I sample from a gaussian with the mean output from step 2
Apply the command and observe the reward (1/L2 distance to target)
collect N steps of experience, compute advantage and old log probabilities,
train actor and critic

My question is this. I have my old log probabilities (probabilities of the actions taken given the means generated by the actor network), but I dont understand how the new probabilities are generated. At the onset of the very first minibatch my new policy is identical to my old policy as they are the same neural net. Given that in the model.fit function I am passing the same set of observations to generate 'y_pred' values, and I am passing in the actual actions taken as my 'y_true' values, the new policy should generate the exact same log probabilities as my old one. The only (slight) variation that makes the network update is from the entropy bonus, but my score
np.exp(new_log_probs-old.log_probs)  is nearly identically 1 because the policies are the same.
Should I be using a pair of networks similar to DDQN so there are some initial differences in the policies between the one used to generate the data and the one used for training?

The idea in PPO is that you want to reuse the batch many times to update the current policy. However, you cannot update mindlessly in a regular actor-critic fashion, because your policy might stray too far away from the optimal point.
This means you repeat your step 6. epoch amount of times for the same batch of trajectories. Usually epoch is somewhere between 3 and 30 but it is a hyper-parameter you need to adjust. For the first repeat, the old and the new policy are the same, so their ratio should be 1. After the first update, the new probabilities will change due to the updated policy, whereas you will still need to use the old probabilities giving you a ratio different from 1. The old probabilities will stay the same during these epoch update steps, whereas your new probabilities will keep changing.


At what point are AWS GPU instances worth it compared to CPU, price-wise? Let's say for:

Image tasks
Deep RL in high dimensional state space

The answer is it depends.
Are you referring to GPU vs CPU when training, or running inference?
When you say CPU is it an x86 CPU or ARM CPU?
What algorithms are running?
Let's look at image classification. Suppose you select a Convolutional Neural Network(CNN) as your algorithm. CNN's were created in the 1980s, but only really saw use when GPU's were used. In 2004 K. S. Oh and K. Jung showed that a GPU was 20x faster than a CPU. In 2005 Steinkrau et al showed a 3x speed increase.
Dan Ciresan et al showed in 2010 and 2011 that a GPU can be up to 60 times faster than a CPU. They used the MNIST handwritten digits benchmark. A simple google search of performance CPU vs GPU will result in many papers that show how much faster a GPU is compared to a CPU.
In this paper Eric Lind and Avelin Parnigoso compare the performance differences between a CPU and a GPU using TensorFlow. In their study they used a Core i7-6500 2.5 GHz CPU and an Nvidia GTX 1070 GPU. In their study they used three CNN AlexNet, Text Classifiction, and the Mnist Digit Classification.
Eric and Avelin showed that GPU outperforms CPU when training, and that during inference the GPU can be slower. In their study AlexNet for example ran for 4 000 000ms on a CPU and under 100 000ms on a GPU. That's a 40x difference in training performance. Converting this to cost. Eric and Avelin did not mention how much RAM.
I am going to make some assumptions and select the following instance types from AWS for training:

ml.g4dn.xlarge [GPU Accelerated] \$0.736/hour [4 vCPU, 16 GiB RAM]
ml.c5n.xlarge \$0.302/hour [4 vCPU, 10.5 GiB RAM]
ml.c5n.2xlarge \$0.605/hour [8 vCPU, 21GiB RAM]

The c5n's are powered by Intel 3.0 Ghz Xeon CPUs and the g4dn feature Nvidia T4 GPUs.
Based on this GPU Accelerated will run at less than 100 000ms or 100s while non GPU accelerated will run at 4 000 000ms or 4 000s. The result is:

ml.g4dn.xlarge - estimated cost \$1.226
ml.c5n.xlarge - estimated cost \$20.133
ml.c5n.2xlarge - estimated cost \$40.333

Based on the experiments showed by Eric and Avelin the CPU instance must be 40x less in cost to be on par with a GPU instance. As you can see the standard CPU is 20 and 40 times more expensive than the GPU. The thought experiment above is based on the paper times for training, not inference, and naturally to get a better answer should be benchmarked.
However, AWS has released their own ARM based CPU's designed specifically for training, and inference.

AWS Graviton Processor
AWS Inferentia
AWS Trainium

Both the Inferentia and Trainium chips offer 30% higher throughput and 45% lower cost per inference than Amazon EC2 G4.


Does DQN generalise to unseen states in the case of discrete state-spaces? In my understanding, DQN is useful because it utilises a neural network as a q-value function approximator, which, after the training, can generalise to unseen states.
I understand how that would work when the input is a vector of continuous values, however, I don't understand why DQN would be used with discrete state-spaces. If the input to the neural network is just an integer with no clear structure, how is this supposed to generalise?
If, instead of feeding to the network just an integer, we fed a vector of integers, in which each element represents a characteristic of the state (separating things like speed, position, etc.) instead of collapsing everything in a single integer, would that generalise better?

An environment is said to have a discrete state-space, when the number of all possible states of the environment is finite. For example, $3\times3$ Tic-tac-toe game has a discrete state-space, since there are 9 cells on the board and only so many different ways to arrange Os and Xs.
A state-space can be discrete regardless of whether integers or non-integers are used to describe it. For example, consider an environment where a state is represented with a single number. If the set of all possible states is $ \{0, 0.3, 0.5, 1\}$, your state-space is discrete, because there are only $4$ states. However, if all possible states is the set of real numbers from $0$ to $1$, than it's not discrete anymore - because there are infinitely many of them. State-space can still be discrete, even if possible states are represented with multiple numbers. For example, our environment could be $10\times10\times10$ cube, where the agent is only allowed to stand on integer coordinates. In this scenario, there are $1000$ different places where the agent can be and hence the state-space is discrete.
The Deep Q-Network can be designed to accept any type of input; just like a regular ANN, it's not restricted to only one integer. An input to DQN is the state of the environment, regardless of how it's represented. For the previous example, you can setup the DQN to have an input layer with $3$ neurons, each one accepting an integer that describes agent's position along the $x, y, z$ axes.
One downfall of Q-learning is that when the environment has a very large number of states and actions, representing each state-action pair becomes impractical in terms of memory. Think of chess, where there are so many different possible positions, and multiple available moves in each of them. Moreover, for an agent to learn properly, every state-action pair value must be visited (agent needs to determine Q-values), which can impractical in terms of training time.
DQN algorithm takes care of these problems: it only needs to store the neural network (also few other things if you use a variation of DQN), and it doesn't need to visit every state-action pair to learn. They way it learns is by adjusting the weights and biases in the network in order to approximate the optimal policy. Given the algorithm is implemented correctly, the agent should be able to pick up some useful patterns (or solve the environment).
I used this paper as a reference for one of my projects. It implements the DQN algorithm to learn to play Sungka (a game similar to Mancala), which has finite number of possible states and actions.


In Deep Deterministic Policy Gradient, are all weights of the policy network updated with the same or different value? I'm trying to understand the DDPG algorithm shown at this page. I don't know what should the result of the gradient at step 14 be.

Is it a scalar that I have to use to update all the weights (so all weights are updated with the same value)? Or is it a list with a different values to use for updating for each weight? I'm used to working with loss functions and an $y$ target, but here I don't have them so I'm quite confused.

Each Q output is a scalar, so the sum of all those is a scalar. Thus, you're taking a gradient wrt your parameters of a scalar. The result is a vector with one entry per parameter.


Is there a good website where I can learn about Deep Deterministic Policy Gradient? Is there a good website where I can learn about Deep Deterministic Policy Gradient?

Spinning Up by Open Ai.
Be sure to read up Part 3 (Intro to Policy Optimisation) before you move on to : https://spinningup.openai.com/en/latest/algorithms/ddpg.html


What kinds of problems can AI solve without using a deep neural network? A lot of questions on this site seem to be asking "can I use X to solve Y?", where X is usually a deep neural network, and Y is often something already addressed by other areas of AI that are less well known?
I have some ideas about this, but am inspired by questions like this one where a fairly wide range of views are expressed, and each answer focuses on just one possible problem domain.
There are some related questions on this stack already, but they are not the same. This question specifically asks what genetic algorithms are good for, whereas I am more interested in having an inventory of problems mapped to possible techniques. This question asks what possible barriers are to AI with a focus on machine learning approaches, but I am interested in what we can do without using deep neural nets, rather than what is difficult in general.
A good answer will be supported with citations to the academic literature, and a brief description of both the problem and the main approaches that are used.
Finally, this question asks what AI can do to solve problems related to climate change. I'm not interested in the ability to address specific application domains. Instead, I want to see a catalog of abstract problems (e.g. having an agent learn to navigate in a new environment; reasoning strategically about how others might act; interpreting emotions), mapped to useful techniques for those problems. That is, "solving chess" isn't a problem, but "determining how to optimally play turn-based games without randomness" is.

I was hoping to see more answers here, but I'll get us started with some examples:

Combinatorial Search Problems: If your problem can be phrased as movement through a combinatorial graph, you don't need a neural network. In particular, your problem should have discrete states, a clear set of actions that are possible in each state, a clear definition of where we start, and a clear definition of what the goal state looks like. The most effective general purpose technique is iterative deepening search. If you have an idea about which moves might be more effective, or better, a function that estimates how far each state is from the goal, you may be able to build a heuristic function and use A* search instead. Common applications for these techniques include pathfinding in video games (or directions in other applications), AI planning, and Automated Theorem Proving.

I'll add some more topics later, but I suspect others have expertise to share here. Let's see some more ideas!


Can I filter barking sounds on the television? My dog goes bonkers every time the sound of a barking dog is heard on a television program.  I never noticed this before but literally every movie or show with an outdoors setting eventually includes the sound of a barking dog. Is it possible to develop a real-time filter that blocks or masks these sounds?

OK, here is one approach.

Acquire a data set of 'clean' audio samples without barking dogs and an data set of barking dogs sounds.
Generate a training set by mixing random selections of clean audio with a random selections of barking dog noises at various volume levels. This is your input data. Your output data is the clean audio before you added the barking.
Find a suitable neural network. This might be a good starting point: https://github.com/andabi/music-source-separation
Train the network.
Apply the network to a load of the clean+bark mixes and check the result by ear. If the results are not good enough, go to 3 to tweak the network. If the results are still poor go to 2 and generate more training data, perhaps adding some distortions or other effects the the barking. Happy with the result? Go to 6.
Publish the results online for the betterment of humanity. (And/or write the network to  a miniature GPU board (such as the NVIDIA Jetson https://developer.nvidia.com/buy-jetson) and work out how to wire that up to the decoded audio in your telly before it heads to the speakers), then blog about it because that would be a great project.


How do two perceptrons produce different linear decision boundaries when learning? I've learned that you can use two perceptrons to ultimately create a classifier for non-linearly separable data. I'm trying to understand how / if  these two perceptrons converge to two different decision boundaries though.

Source: https://tdb-alcorn.github.io/2017/12/17/seeing-like-a-perceptron.html

I don't understand how the second perceptron creates a different decision boundary when it has the same input as the first perceptron? I know the weights can be initialized differently but does this second perceptron classify something else? Shouldn't the decision boundaries be converge to be the the same ultimately after training?

Source: https://tdb-alcorn.github.io/2017/12/17/seeing-like-a-perceptron.html

I think your confusion comes from the fact that you are calling those two hidden nodes "perceptrons". You shouldn't call the hidden nodes in your network perceptrons. You should call them "nodes" or "neurons", although the term "multilayer perceptron" comes exactly from that.

You should think of a perceptron as something like

A perceptron simply computes a linear combination of the inputs using the corresponding weights. A single perceptron can (only) linearly separate a set of points.

However, if you have two "stacked perceptrons" (like the hidden nodes in your network), then, provided that both "perceptrons" (or hidden nodes) only compute a linear combination of the inputs, the output of each hidden node can be interpreted as line (like the violet and green lines in your picture).

  I don't understand how the second perceptron creates a different decision boundary when it has the same input as the first perceptron?

This is because the two perceptrons will eventually have different weights. You can see from your diagram that $p_0$ has weights $w_{0, 0}$ and $w_{0, 1}$, whereas perceptron $p_1$ has weights $w_{1, 1}$ and $w_{1, 0}$. Note that weights are learned, during training, depending on your loss function (at the output node $p_2$) and the labeled dataset (that you use to train that simple neural network).

  I know the weights can be initialized differently but does this second perceptron classify something else?

If you consider the two hidden nodes (in your network) separately, then (as I said above) the output of these two hidden nodes can be interpreted as a line, and these lines might be different (as illustrated in your picture), because these two hidden nodes will eventually learn something different about the input. However, you should also think of the output of two hidden nodes as contributing to the input of the output node. So, in a certain way, you're combining the decisions of the two hidden nodes (or, as you call them, "perceptrons").

Note that you will need to introduce "non-linearity" in those hidden perceptrons in order to be able to non-linearly separate a set of points.

  Shouldn't the decision boundaries both converge to the same ultimately after training?

No, not necessarily. The decision boundary found by the perceptron learning algorithm depends on the initialization of the weights and the order that the inputs are presented. See chapter 4 (specifically, pages 192-196) of Pattern Recognition and Machine Learning by C. Bishop. Two perceptrons would be guaranteed to converge to the same solution if the loss function was convex (i.e. there's only one optimum).


Is it legal to construct a public image database (for deep learning) with images from the internet? I am trying to put together a public agricultural image database of corn and soybeans, to train convolutional neural networks. The main method of image collection will be through taking pictures of various fields in the growing season. The images will be uploaded to a public data sharing site which will be accessible by many.

However, I could get much more images compiled if I were to take some off of, say, Google Images. If there anything wrong with this? Would there be any issues with copywright infringements if I find the images on a publicly-available search engine? I need a lot of images, so I thought this would be a good method of increasing my image numbers.

I believe this is covered under Section 107 of the Copyright Act states:

the fair use of a copyrighted work, including such use by reproduction in copies or phonorecords or by any other means specified by that section, for purposes such as criticism, comment, news reporting, teaching (including multiple copies for classroom use), scholarship, or research, is not an infringement of copyright.

This section was intended to enable utilization of copyrighted material where that utilization is in the public interest. As long as you stay within the bounds as stated above you should not be considered in violation of a copyright.


How can I implement a GAN network for text (review) generation? How can I implement a GAN network for text (review) generation?

Please, can someone guide me to resource (code) to help in text generation?

As @Clement mentions, text_gen_description gives a good overview!, but the paper seqGAN paper describes the REINFORCE approach more in depth, as they are the first to do it (i believe). This is probably the approach most take now of days when going the GAN route.

Note that just basic MLE training has shown promise with openAI's GPT2. When i need a text generator, fine tuning one of the provided models is usually my goto.

Also if your looking for seq gans code base (you asked for example code) here is is: git repo

Good Luck!


Method for Multi-class/category? I am having issues getting started with a multi class problem with multiple features and hoping someone could please point me in the right direction.

I have data that is structured like this for training:

Item State Code1 Code2 Code3 Route
---  ---   ---   ---   ---   ---
item1 MI   A1    33    blue  Route1
item2 TX   A3    35    yellow Route2
item3 NM   A4    36    green  Route3
item4 NM   A4    37    green  Route3

Essentially I am trying to figure out where to even start. The goal is to know where to route the Items based on the features State, Code1,2, and 3. The route is dependent on a mix of the codes and state, and I want to build a model that says when I have code X, Y, Z and Color XX, then it is probably route 1 (some routes of course in the training data might have X, Y as codes and a different Z)

I am assuming I will need to one-hot encode the features like the State and codes? But from there does anyone know of which type of model I should go for? I would assume a Neural Net of some kind, I've explored CNN's and Random Forrest.

My money would be on something much simpler like Naive Bayes.

In my experience for small data NB outperforms the more exotic methods.

Also, if you want to get more value out of your training data, try 10 fold cross validation