Influence of location on a Neural Network trained for parking detection occupancy I loaded a neural network model trained with Caffe by other people in OpenCV.

The model should detect the presence of a car in a single parking spot outputting the probability of it being free/occupied.

The model was trained with images all belonging to the same parking area, taken at different hours of day and with different light conditions. Images were taken by different cameras but the cameras are all of the same model (raspberry cameras).

I tried to run the model with a few images some of them taken from their dataset and other downloaded from google.

The images taken from their dataset are correctly classified while the ones taken from google are not correctly classified.

My question is: is it possible to deploy a NN model trained with images all coming from a single parking area in another parking area? Is not such a model for parking detection occupancy supposed to generalize independently from the location where training images have been taken?

If you know about an already existing trained model that works good please let me know.

My question is: is it possible to deploy a NN model trained with images all coming from a single parking area in another parking area? Is not such a model for parking detection occupancy supposed to generalize independently from the location where training images have been taken?

Generalisation in machine learning has limits, and for complex systems such as computer vision, it is very easy to exceed them. This is why the state-of-the-art systems use huge amounts of training data which is often augmented with automated variations, plus trained on for many days.

If you want some factor in your model to generalise well, you should:

a) Train with variations in that factor

b) Test with variations in that factor that have not already been seen in the training set

For the factor you wish to have the model generalise in, then it looks like (a) has not been done, and that you have just done (b) and found that the first attempt has failed an important test that you care about.

I suspect the trained model you have used will generalise well by time of day, type of vehicle and some other details. It does not appear to generalise to a new location because it was trained on only one location.

If you have limited training data, you may want to look at approaches to fine-tuning against more general image classifiers (such as the many ImageNet-trained classifiers, which are available to download). However, even then you should train with a few varieties of parking space, or at the very least with the spaces that you care about (if you do the latter then it probably won't generalise to other spaces, but you won't care).

It may also be worth seeing what you can do to transform the input that you are working with to have similar view over the new parking area that the training images have. Using a transform to align the outline of the new area with the old one, and possibly some colour correction could help a lot.


Is there an LSTM-based unsupervised learning algorithm to label a dataset of curves? I have a big amount of light curves (image below).

I am trying to label the points as signal or background (the signal appears usually periodically, several times, for a given light curve).

More precisely, I want to identify the downward spikes (class label = 1) from the background (class label = 0).

However, the data is not labeled. I tried labeling it by hand, and using a bi-directional LSTM succeeds in labeling the data points properly. However, there are thousands of light curves and labeling all of them would take very long.

Is there any good unsupervised approach to do this (unsupervised LSTM maybe, but any other method that might work on time series would do just fine)?

You can try to work with Gated Recurrent Units or GRU. This will solve your problem of with too much latency time that LSTM required. LSTM also doesn't give preference on newer data too. For more information, you can follow more on this great article

GRU Cells


How do you handle multiple categorical values in a single column for wide_deep model in tensorflow? To start, let me just say that I am very new to tensorflow and Machine Learning in general. But, as part of my learning project I am trying to adapt the tensorflow wide and deep model to generate movie recommendations. However, the part I'm getting stuck on is handling multiple values for a categorical column. Below is a sample of how a couple of rows in my CSV look like.

    ID,Genres,Tags,Rating,Recommendations
    ----------------------------------------
    1,genre1:genre2:genre3,tag1:tag3,4.3,44
    2,genre2:genre3,tag1:tag5,3.7,22

The Genres and Tags column have multiple categorical values. I have looked at tf.string_split to parse the strings and return a SparseTensor.

Once I have parsed my delimited string values into SparseTensor, what do I do with? If I want to create categorical_column_with_vocabulary_file, how does the SparseTensor interact with it? Is that even the correct step? Should the SparseTensor be converted into a something before I can create a categorical_column_*?

I am just not sure how to train the Wide and Deep model when you have multiple categorical values for a single column. Some people have suggested that I use each 'Genre' as its own column and encode it using One-Hot, but it is not realistic for me to do this, because there could be 100s of genres and tags in the data.

Any help on this matter would be welcome. Thank you!!

I am also stuck with this problem and after some searchs I end up here.

Maybe you can take a look at indicator column. I think embedding column behaves in a similar way.


Initialising DQN with weights from imitation learning rather than policy gradient network In AlphaGo, the authors initialised a policy gradient network with weights trained from imitation learning. I believe this gives it a very good starting policy for the policy gradient network. the imitation network was trained on labelled data of (state, expert action) to output a softmax policy denoting the probability of actions for each state.
In my case, I would like to use weights learnt from the imitation network as initial starting weights for a DQN. Initial rewards from running the policy are high but start to decrease (for a while) as the DQN trains and later increases again.
This could suggest that the effect of initialising weights from the imitation network has little effect, since it kind of undo's the effect of initialising weights from the imitation network.

My understanding is that you are first training a policy network using imitation learning. Then you are adjusting that trained network in some way to be a value network for DQN. The most obvious change would be to remove softmax activation whilst keeping the network layer sizes identical. This would then present Q estimates for all actions from any given state.
The initial estimates would not be trained Q values though, they would be the "preferences" or the logits for probabilities to support a near optimal action choice. The main thing that will be likely in the new network is that for the one near optimal action choice, the network would predict the highest action value. As you derive the target policy by taking the maximising action, initially this looks good. However, the problem is that the Q values that this network predicts can have little to no relation to the real expected returns experienced by the agent under the target policy.

Initial rewards from running the policy are high but start to decrease (for a while) as the DQN trains and later increases again.

I think what is happening is that initially the greedy policy derived from your Q network is very similar to the policy learned during imitation learning. However, the value estimates are very wrong. This leads to large error values, large corrections needed, and radical changes to network weights throughout in order to change the network from an approximate policy function to an approximate action value function. The loss of performance occurs because there is not a smooth transition between the two very different functions that also maintains correct maximising actions.
I don't think this can be completely fixed. However you might get some insight into potential work-arounds by considering that you are not just doing imitation learning here. Instead you are performing both imitation learning (to copy a near optimal policy) and transfer learning (to re-use network weights on a related task).
Approaches that help with transfer learning may also help here. For instance, you could freeze the layers closer to input features, or reduce the learning rate for those layers. You do either of these things on the assumption that the low-level derived features (in the hidden layers) that the first network has learned are still useful for the new task.


When the AI singularity takes over, what will there be left for us to do? Since the first Industrial revolution machines have been taking the jobs of people and automation has been a part of human social evolution for the past 3 centuries, but all in all these machines have been replacing mechanical, high-risk and low-skill jobs such as a production line of an automobile factory.

But recently with the advent of computers and the improvement of AI, and the quest to find a Singularity (that is, a computer capable of thinking faster, better, more creative and cheaper then a human being, capable of self-improving), our future will lead to the replacement of not only low-skill workers, but high-skill as well. I'm talking about a future not too far when AI and machines will replace artists, designers, engineers, lawyers, CEO's, filmmakers, politicians, hell even programmers.
Some people get excited by this, but honestly, I get somewhat scared.

I'm not talking about the money issue here, although I'm not a fan of the idea, let's suppose the universal income has been implemented, and suppose it works fine. Also not talking about the "Terminator's world where machines will wage war against humans", let's suppose too they are completely friendly forever.

The issue here is the one of motivation for us humans. When the AI singularity takes over, what will there be left for us to do? Every day, all day long?

What are we going to do with our lives? Suppose I love to paint, how can I live my dream of becoming a painter if the computer makes better art then I will ever be able to do? How can I live knowing that no one will care about my paintings because they were made by a mere human? Or the real me for example (I, Danzmann), I love to code, learned my first programming language with 9 years old and been on it ever since then, it looks sad to me that in some years I may never touch on that again. And that goes for all the professions, everyone is passionate about something, and with the singularity, every single one of them would just have to cease to exist.

So, what are we going to do in this future? What am I going to do? Play golf all day, every single day for the rest of my life (A Hyperbole figure of speech, but you get my point)?

Also, what is going to be the motivation for my children? What am I going to tell them to go to school? When someone asks "what do you wanna be when you grow up?", and the inevitable answer is nothing.

If highly advanced AI takes control of all scientific research, then what is the reason for us to learn? What is the reason that us humans would need to dedicate decades of our lives to learn something if that knowledge is useless, because there are no more jobs and the scientific research is done solely by AI?

Given all your assumptions about AI turn out to be true, we would have some kind of utopia, where no one has to work, and there is plenty of everything. Fair enough. Your other assumptions is about human nature, and that is where I'd challenge your conclusion: Just because there are computers better than humans at some task, that does not automatically take all enjoyment from doing it.

I have three arguments in favor of my stance.

There already are computers better than humans at checkers, chess, backgammon, starcraft, mastermind, go,.. and many more. Yet these games still get played, even if no human can hope to ever be as good as a computer.
There will be domains where the evaluation of quality is so vague or personal, that the notion of being "better" is useless in any objective sense. I am thinking mainly about art. Playing into argument one, photo cameras are already better "painters" of reality than humans can ever hope to be, yet people still paint. And their paintings gets appreciated, even the photo-realistic kind.
I'd say that the whole mindset of "if someone is better than me at x it is not worth doing x" will have outlived its lifetime very shortly. Society didn't always spin that way I feel, it is largely due to the influence of the North American way of life to always strive to be #1 and everything below that being trash. Globalization already puts this way of thinking at risk, with many young people being disillusioned or even depressed because, flippantly put, "whatever you do, there is always an Asian kid who does it 10 times better". We don't have to wait for AI to outshine us, the rest of the world already does. As a consequence, we need to adapt our way of approaching that fact, stop seeing it as diminishing to our worth, and move on.

As a closing note, I also see it as a problem to the general population that within a short time we essentially have to change a very substantial part of our world view, AI being better than us, us not having to work anymore etc. All the economical revolutions in the past, the neolithic, the industrial, and most recently, the digital, had a longer transitional period where people could grow accustomed to the new world. And even with that transition it was hard enough for many people. Yet, most dealt with it, and later generations can hardly imagine a world where the new change doesn't exist yet, and I personally don't see why the next revolution should be any different.


How can object types be differentiated in the input of a neural network? In a neural network when inputting nerve input to sense a 2D environment, how do you differentiate two types of objects (with similar shape and size) so the neural network can treat them differently?

Each neuron in the input layer of a neural network essentially gets 1 dimensional input (range between two values) but 2 dimensional input would be needed to send both collision and category/type information through each input layer neuron. How do you get around that?

Note: After having confusion regarding the scenario / situation I'm asking about compared to other more complex scenarios, and the long comment series that ensued, I'm realizing one challenge of this site is that it's much more complicated and diverse subject matter than code, or the various other topics of Stack Exchange where the problems can be very clearly and simply expressed. Here it's more challenging to express your question and scenario clearly to avoid confusion.

Also there's probably a higher skill gap between an AI learner / enthusiast, and an expert AI specialist, compared to other fields, so that could potentially lead to even more difficulty communicating the answer / question in ways everyone can understand without confusion. Challenging SE site to ask good questions on!

Neural networks learn. That's what they are for. For your task there are two sensible scenarios:

You have a fixed reaction for danger and a fixed reaction for food and you only have to learn how to distinguish between them. In that case you basically try to classify the situation to trigger the right fixed response and this classification would be learned by backpropagation.
You directly learn to act for a given situation. In that case you can either use a genetic algorithm or you use reinforcement learning with backpropagation.

I would recommend using a genetic algorithm, because it is significantly easier and also makes sense in this situation. You would randomly initialise your network, let it run around in the environment and remember how much food it ate and how often or how quickly it died. Then you would randomly change the weights of your network and do the same thing again. If it did better this time around you would proceed to use the new weights otherwise you go back to the old weights and try a different random change.
By selecting successful random changes it would over time learn to avoid danger and seek out food.

Edit: To my mind you have a fundamental misunderstanding how perception works. If you see a lion and a cake, do those trigger different kinds of cells on your retina? No! All nerve cells are used to detect all kinds of objects! The classification, i.e. whether you are seeing a lion or a cake happens in the neural network i.e. in the higher regions of your visual cortex, far removed from the initial nerve activation. Your lion might be yellow and your cake might be yellow, only if you analyse the high level structure of your nerve inputs can you decide what you are seeing. That is the task of a neural network. And that high level structure analysis is what a neural network learns.

What seems to confuse you is the example you linked. In that example this very sparse distance measuring is enough to differentiate between walls and boosters in your high level structure analysis, because the different points of the walls that you sample have a certain relative position that you can analyse and conclude that they constitute the wall.

In your scenario very sparse distance measuring will not help you obviously. The distance of an object doesn't tell you whether it's a lion or a cake. Distance and color would be a solution to that. Or, more realistically, you have different shapes and much tighter distance sampling, and high level analysis can work out the shape from a couple of closely measured distances.


Can residual neural networks use other activation functions different from ReLU? In many diagrams, as seen below, residual neural networks are only depicted with ReLU activation functions, but can residual NNs also use other activation functions, such as the sigmoid, hyperbolic tangent, etc.?

The problem with certain activation functions, such as the sigmoid, is that they squash the input to a finite interval (i.e. they are sometimes classified as saturating activation functions). For example, the sigmoid function has codomain $[0, 1]$, as you can see from the illustration below.

This property/behaviour can lead to the vanishing gradient problem (which was one of the problems that Sepp Hochreiter, the author of the LSTM, was trying to solve in the context of recurrent neural networks, when developing the LSTM, along with his advisor, Schmidhuber).
Empirically, people have noticed that ReLU can avoid this vanishing gradient problem. See e.g. this blog post. The paper Deep Sparse Rectifier Neural Networks provides more details about the advantage of ReLUs (aka rectifiers), so you may want to read it. However, ReLUs can also suffer from another (opposite) problem, i.e. the exploding gradient problem. Nevertheless, there are several ways to combat this issue. See e.g. this blog post.
That being said, I am not an expert on residual networks, but I think that they used the ReLU to further avoid the vanishing gradient problem. This answer (that I gave some time ago) should give you some intuition about why residual networks can avoid the vanishing gradient problem.


Additive Attention in Convolutional Networks Attention has been used widely in recurrent networks to weight feature representations learned by the model. This is not a trivial task since recurrent networks have a hidden state that captures sequence information. The hidden state can be fed into a small MLP that produces a context vector summarizing the salient features of the hidden state.

In the context of NLP, convolutional networks are not as straightforward. They have the notion of channels that are different feature representations of the input, but are channels the equivalent to hidden states? Particularly, this raises two questions for me:

Why use attention in convolutional networks at all? Convolutions have shown to be adept feature detectors––for example, it is known that higher layers learn small features such as edges while lower layers learn more abstract representations. Would attention be used to sort through and weight these features?
In practice, how would attention be applied to convolutional networks? The output of these networks is usually (batch, channels, input_size) (at least in PyTorch), so how would the attention operations in recurrent networks be applied to the output of convolutional networks?

References

Convolutional Sequence to Sequence Learning, Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, Yann N. Dauphin, 2017

It seems that in the reference you provided they use attention to compute weights for the encoder representations based on the context provided by the decoder (Fig. 1). As far as I can tell, attention is applied after convolution (in fact, after the GLU step), so it does not affect the feature maps directly. Rather, attention is used to select the target words in the decoder. In other papers (e.g., this one), attention is applied directly to the feature maps in a way that is more similar to what you described.

Regarding your second question, the paper you reference actually provides a link to the source code they used. It is written in Lua (using Torch) instead of PyTorch, probably because PyTorch development was just starting when the paper was published. At any rate, you should be able to follow the Lua code and translate it into PyTorch.


why the sigmoid function will be 1 and 0 if we use a fully connected layer that produce a big enough positive(res negative )output I am using a fully connected neural network that uses sigmoid activation function. If we feed a big enough weights the sigmoid function will finally become 1 or 0, is there any solution to avoid this?
Will this lead to classical sigmoid problems vanishing gradient or exploding gradient?

In general it's better to not use sigmoid function in any hidden layer. There are many other great options such as ReLU and ELU. However, if for any reason you have to use sigmoid-like function, then go with Tanh function, at least it has ~0 mean.


What happened after the second AI winter? I know this is a very general question, but I'm trying to illustrate this topic to people who are not from the field, and also my understanding is very limited since I'm just a second-year physics student with a basic understanding of R and Python. My point is, I'm not trying to say anything wrong here.

So according to Wikipedia, after the second AI winter, which happened because expert systems didn't match expectations of the general public and of scientists, AI made a recovery "due to increasing computational power (see Moore's law), greater emphasis on solving specific problems, new ties between AI and other fields (such as statistics, economics and mathematics), and a commitment by researchers to mathematical methods and scientific standards".

What I'm trying to understand now is whether the rise of AI is rather connected to greater computational power available to the public or whether there have been fundamental mathematical advances that I'm not aware of. If the latter is the case (because according to my understanding, the mathematical models behind neural networks are rooted in the 70s and 80s), I would appreciate examples.

Again, please don't be offended by the general character of this question, I know it is probably really hard to answer correctly, however, I'm just trying to give a short historic introduction to the field to a lay audience and wanted to be clear in that regard.

My reading of AI development (somewhat simplified here) is that the availability of large data sets, increased computing power, and the introduction of new machine learning algorithms (which require large data sets and massive computing power) contributed to the resurgence of AI.

However, as witnessed on this site, there has been a paradigm shift within the field: while previous approaches to AI were largely symbolic, with a bit of connectionism thrown in, the current AI mainstream is purely based on statistical models and machine learning.

Massive (distributed) computing power on its own is not sufficient, as there was a bottleneck with domain modelling/knowledge acquisition in traditional symbolic approaches. New algorithms (basically further developments of neural networks) on their own would also not be sufficient without massive amounts of training data. Only the combination of all three elements enabled the AI resurgence.


What is the best way to integrate unchangeable ethics into a chatbot I am building a generative model chatbot as a research and learning project. One of the most important parts of my project is to research ways in which I can make this chatbot work in a consistently ethical fashion.

This chatbot is simply a single Seq2Seq network running on my local machine. It can't be interacted with over the internet (yet), although I may end up creating a way to do that. It has no feedback loops of any kind as of right now, though reinforcement learning with a loop might be helpful.

The idea is that there would be some sort of unchanging knowledgebase for the chatbot to use that has hard-coded ethical statements and values that the bot has no ability to change. Whenever a question is asked of the chatbot,before being inputted to the network, the knowledgebase is searched and relevant facts will be appended to the input (separated from regular inptu by  tokens.

My question is, will this even be effective at allowing it to generate its own responses yet still be confined to the ethical standards given it?

My main concern is that it may begin to ignore these "facts" over time, and they may become irrelevant.

Another (possibly much better) approach might be to use deep reinforcement learning. However I may find it difficult to implement with my existing Sequence-to-Sequence network.

So which would likely be better? Or perhaps I should try a combination of the two?

Basic Fairness
If by ethics the question means that the system design is to foster basic ethics in the context of a conversation, then at least three attributes can be designed into the system.

Egalitarianism — no favor or disregard on the basis of gender, age, beliefs, financial status, ancestry, or culture, even if those features of the conversant are known and useful to know in personalizing the conversation
Temporal fairness — ensuring that the interaction with new correspondents and veteran correspondents provide these with equal access to relevant information, even if the establishment of trust is an evidence based process controlled by the system (recommended)
Sharing — balancing the objectives of the system to serve the interests of others as much as the interests of the system and its stakeholders — The Golden Rule: Treat others as you wish to be treated.

These attributes can be encoded in fuzzy rules, incorporated into the value function of an ongoing learning algorithm, or expressed in the loss function used during training, directing what is learned by a network.  All of these strategies require the tracking of metrics and the uses of them in adjusting behavior, a kind of social awareness.
There are obvious ways to do this in the mathematical expressions of values, assets, advantages, affirmations, attentions, losses, errors, pains, or other learning feedback approaches are assembled.  In generative network topologies, a discriminative network would need to be able to recognize unfairness and selfishness using the above principles.  Such places specific requirements on the designs of metrics and how fairness and the balance of interests are quantified.
A sum of squares across the dimensions (features) of potential inequity that can develop during a conversation is a possibility.  Using the fourth power instead of squaring would imbue permissiveness for smaller inequities but a drastic aversion to greater inequities.
$$\ell = \sum {(b_i - b_j)}^4 + \dots$$
The variable $\ell$ is the loss function result and $b$ is the benefit to conversation participants $i$ and $j$ where the two might be a human and the chatbot, a human and a stakeholder of the chatbot production server, or two humans that are in separate conversations with the chatbot, or two humans that are in the same conversation with the chatbot.  Note the ellipsis indicates more terms that affect loss as needed.
Cognitive Ethical Sophistication
If by ethics the question means the kind of ethics passed from families that value honor and truth to their children or taught at Harvard Law, the system design may have to wait until cognitive abilities are further developed in computing machinery.  Otherwise the question author will have to further AI to achieve the ethical sophistication.
Research into semantic acquisition, abstraction, automated assembly of causal models, application of these abstractions and models in planning and execution, and the integration of these with natural linguistics is ongoing.  Broad cognition and What some might call wisdom escapes the products of AI as of this writing.
A sufficiently deep network may approximate dialog that demonstrates what appears to be ethical awareness if trained with data that represents stories and conversational actions deemed to be ethical in those stories.  However, compacting learning the layers of comprehension that humans acquire through books, movies, other media, family life, and community events over a few decades (if they acquire it at all) into a computing project may be an overoptimistic objective at this time.
Although a collection of classic works might demonstrate much of a high ethical standard, it is not labeled data.  One would need to identify which character was transformed in a way that demonstrates the valuable ethical standards and then configure the learning process to learn from the character arcs of ethical characters in the stories.
Inherent Feedback

It has no feedback loops of any kind as of right now, though reinforcement learning with a loop might be helpful.

There is no way to adapt without feedback.  If the system is adapting, some feedback mechanism is correcting conditions that are maladaptive, even if the mechanism is opaque to the observer or identified by some other name.  In the case of GANs, the feedback creates a balance through the discriminative and generative network components.
$$\Bigg(\Big((G, E_1) \Rightarrow D\Big) \Rightarrow G, E_2\Bigg) \Rightarrow D \dots$$
That does not exclude the possibility that additional feedback paths may provide additional design advantages.
Hard Coding Ethics
Can judicious behavior be hard coded?

The idea is that there would be some sort of unchanging knowledge base for the chatbot to use that has hard-coded ethical statements and values that the bot has no ability to change.
Will this even be effective at allowing it to generate its own responses yet still be confined to the ethical standards given it?

It is not unchanging knowledge in the domain of ethics that leads to justice, but unchanging principles that guide the detection of what is not ethical that guides a continuous improvement of ethical standards.  That is why even constitutions have rules for amendment.  In probabilistic terms, without the balance of doubt and certainty, no ethical growth can occur.
No one would read a book or watch a movie where the protagonist has no character arc because it is unremarkable.  When a story is remarkable it is because ethics are adaptive and the adaptation must be a new kind that is not already covered many times in previous books or movies.  People like to be surprised by these adaptations.  They must occur against all fears and opposing external forces that are trapping the protagonist in a prison of mediocrity.
The story climax is when the protagonist, against all the building forces that keep them unremarkable, takes a path that discards all previous planning and forges a new radically different but fundamentally better one.
Excellent Objective in the Question
The intention to develop a system that is ethical is honorable and ethical in itself, so the question author may imbue some of those qualities into the system being designed.  In my opinion, such would be a triumph of greater value to the field of AI than all other progress up until now.
To make such an advancement, the framing of the problem will require thinking in new directions.  Consider the stated concern.

It may begin to ignore these "facts" over time, and they may become irrelevant.

Two principles may prove elucidating in regard to this concern.

Every language in common use has the equivalent of the English words should and ought, indicating a universal sense that something is awry.  This detection and identification of imbalance is central to the definition of heroism and honor, which is making at least an attempt to re-balance it.  Even the Oracle in the Matrix, after creating imbalance, sits on the park bench talking to Sati in the warmth of a new balance.  Identifying what needs disruption and re-balancing and constructing a strategy to do so is the one thing in common across all ethical behavior.  Ethics is remedial.  Some ethical choices require unlearning, and that is the most timeless feature of ethics.
Some of the most ancient principles, such as the Golden Rule or the idea of transparency – that right thought and right action are inescapably coupled – are more pertinent to this project than anything one can find in a book about machine learning.  The computer cannot hide a malicious intention and then converse ethically.  The objectives of the stakeholders of the system will need to also be ethical.

First Decision in Approach
The decision to make is the level of abstraction to apply.  Should this chatbot be ethical or be ethically minded?
The first would require the imbuing of ethical character into the bot.  The second would require the imbuing of the acquisition of ethical character into the bot.  As difficult as the more abstract second choice may seem, the first may be unachievable.  We don't have an example of it.  We do, however, have one proof of concept for the second choice: Good people.
The other sub-question to address is regarding selection of AI components.

So which would likely be better [generative or reinforcement]? Or perhaps I should try a combination of the two?

If you are serious about this objective beyond the simple tracking of basic fairness metrics, then you may have to use elements of both and possibly new things that extend the list of AI components and principles to use.
Take One Step Back
One thing to understand about generative topologies is that they are not really adversarial, even if that word is in the name of a seminal paper.  They are usually highly symbiotic and involve equilibria.  What would the discriminative network be without the generative one?  What would the generative one generate without the discriminative one?
The two collaborate to create a feedback system exhibiting a balanced equilibrium.  This is like biological stasis, which is the genius of it.  It is also like chemistry.  Salt in saturated suspension in water involves both dissolving and crystallization in a constant expression of opposing reactions at a molecular level.  Similarly, all ethical decisions are decisions are made within the context of at least one equilibrium, and the details matter.
Every court case where a judge or jury is to apply the public perception of ethics to the case has two sides presented.  They may appear to be adversaries, but the entire legal process is a sophisticated and symbiotic design to force collaboration in the interest of justice.
Every educated journalist understands the importance of avoiding sensational reporting.  Dramatically presenting only one side of the story sells news and is a constant temptation.  Journalistic integrity demands striking an equilibrium.  The result of the ethical choice determines whether the news item educates the public or adds polarization to a culture that may already be partially crippled by extremism.
Take Another Step Back
Reinforcement is an odd name to give what is essentially Agile planning via the probabilistic projection of expected advantages.  Should lean methods for strategic approaches to conversation be in this ethical chatbot?  If ethical behavior is defined to include telling stories that convey wisdom, then yes.  Roger Schank's work is of great interest in that case.  His work in story based reasoning may be a worthy direction of study.
Adapting the foundations of ethics to the ever changing story inherent in every significant conversation may require planning.  It is possibly unavoidably inherent every conversation when the intention of the participants is to do more than exchange pleasantries.


How to start looking for machine learning opportunities in projects I am a software engineer who has taken an interest in machine learning about a year ago. I work on a couple of projects where a large amount of data is collected, and am wondering what I can do (or practice) in order to start recognizing opportunities to implement machine learning. Until now, I've spent some time building various classifiers with tensorflow, caffe, etc. using pre-defined data sets for random things I've found online (whether it be photos, spreadshets, etc) - and I just want to be able to use it at work. What is the best way to start getting in the frame of mind where I start recognizing where I can apply it? I found this article: https://hbr.org/2017/10/how-to-spot-a-machine-learning-opportunity-even-if-you-arent-a-data-scientist but it didn't seem too helpful to me. If there are any other thoughts and/or resources online that you could point me to, that would be helpful. Thanks.

If you are trying to see whether machine learning can contribute to your work, first thing you ought to do is to see if any of the following tasks could help the business at hand. Machine learning tasks include the following:

Classification
Regression
Clustering
Density estimation
Dimensionality reduction


Do I need to store the policy for RL? I am creating a zero-sum game with RL and wondered if I need to store the policy, or if there are other RL methods that produce similar results (consistently beating the human player) without the need to store the policy and comes the correct decision 'on the fly' - would this be this off-policy?

If your game agent performs any kind of advance learning from self play or database of moves, that will generate parameters for some kind of model (e.g. a table of expected values, or neural network weights to select a preferred action). This is unavoidable, and if you want to re-use the results of that machine learning, you absolutely have to store the parameters somewhere.

  if there are other RL methods that produce similar results (consistently beating the human player) without the need to store the policy and comes the correct decision 'on the fly' - would this be this off-policy?

If your agent can access a model that accurately predicts (or accurately samples) outcomes from actions, it can look ahead and plan from the current state. Typically board games and the like do allow you to have such a model, based on the game rules. Some look-ahead techniques are essentially RL methods applied with a focus on solving a decision "just in time", others are more related to search. Common techniques used in game playing are A* search, minimax search (with alpha-beta pruning for performance improvement), Monte Carlo Tree Search. Used purely with just some pre-coded heuristics and a game model, these search techniques do not require you to store model parameters. The downside is that you must spend more computing resources per game move in order to run the search/planning and drive the policy. However if your game is simple, or your heuristics good, or you don't mind a significant wait per computer move, then this approach can be very effective.

This difference between learning and planning is not directly about on-policy vs off-policy, although planning may be considered off-policy as it assesses many actions, most of which the agent does not take.

What a planning method typically looks like in terms of data is a memory-based representation of a game tree starting from the current position, with internal "scores" very similar to RL value functions used to track consequences of action choices. Search and planning methods each have different ways to prune the tree down to a reasonable size and select which game states and actions to look ahead further in. Using "pure" search allows you to work exclusively with this temporary in-memory representation, discarding it after it selects a next action.

You have probably heard of AlphaGo, AlphaGo Zero etc, which are state-of-the-art game playing agents that learn through self play. These, and similar agents, use a combination of a learned policy (from self-play), plus a look-ahead search which refines it for the current game position, getting the best of both learning and planning. The parameters for any learned policy do have to be stored of course. But the benefit of the combined approach is that you can balance resources placed into general learning of the game (stored as parameters to a value function or policy) and specific choice of action for a current game position, which can be much more focussed.


Other deep learning image generation techniques besides GANs? Can you please point me to some resources about image genereation besides GANs?
Are there any other techniques throughout history?
How did idea of image generation evolved and how it started?

I tried googling "image generation before gans" and similar alternatives but without any success.

There are several generative models that have been proposed before or roughly at the same time of the GAN (2014). For example, the deep Boltzman machine (2009), deep generative stochastic network (2014) or variational auto-encoder (2014).


Couldn't an AI cheat when trying to follow its goal? Hello, I was reflecting about what implications might building a strong AI have and I came across some ideas which I find disturbing, I'd love to have some external thought on that :

1) If we ever managed to create an AI say nearly as smart as a human, It would probably have been programmed with some concrete goals, as the AIs we are programming right now : Reinforcement learning allow an agent to try and increase a "reward" variable, regression is all about getting closer to a certain goal function etc..

But then a strong AI, which would undoubtedly be able to understand how it is built, just as we understand (partly at least) how our brains work, because it would be as smart as its creators and we don't tend to build machines that are as hard to understand as brains.

Then couldn't such an agent figure out the best way to achieve its goals would actually not be, say, pleasing and protecting the humans like we would've wanted it to do, but to get control of its own program and maximize whatever reward it was set to pursue ? Just as we could decide to branch electrods to our brain if we were able to find out how exactly our brain was built.

I really don't see how this scenario could ever be avoided if we were to build such an AI, apart from finding a perfect security preventing anyone from accessing the code of the said AI (including itself).

2) On the same note, I also wondered, could it try to not only satisfy its goals by "cheating" (updating its reward variables for example) but also to change itself, or commit suicide ? After all, we humans have never been able to figure out what goals we were meant to pursue (by that I mean what reward variable in our brain we are trying to increment), and many philosophers reflecting upon that matter thought about death as an escape from once goals. So my question is : could it try to change its code or kill itself ?

I have other questions and thoughts I would like to discuss, but I think this a good start, to test whether I'm in the right place for this kind of discussion.

Looking forward to your thoughts.

These are two good questions. To address the first, this is a known problem in AI safety and is called "Wireheading". There is currently no known solution other than to somehow prevent an AI agent from being able to alter its internal state in this fashion.
If you are interested in this area in general then I recommend this paper as an entry point into the literature.


Why is the hyperbolic tangent with MSE better than the sigmoid with cross-entropy? Usually, in binary classification problems, we use sigmoid as the activation function of the last layer plus the binary cross-entropy as cost function.

However, I have already experienced (more than once) that $\tanh$ as activation function of last layer + MSE as cost function worked slightly better for binary classification problems.

Using a binary image segmentation problem as an example, we have two scenarios:

sigmoid (in the last layer) + cross-entropy: the output of the network will be a probability for each pixel and we want to maximize it according to the correct class.
$\tanh$ (in the last layer) + MSE: the output of the network will be a normalized pixel value [-1, 1] and we want to make it as close as possible the original value (normalized too).

We all know the problems associated with a sigmoid (vanishing of gradients) and the benefits of the cross-entropy cost function. We also know $\tanh$ is slightly better than sigmoid (zero-centered and little less prone to gradient vanishing), but when we use MSE as the cost function, we are trying to minimize a completely different problem - regression instead of classification.

Why is the hyperbolic tangent ($\tanh$) combined with MSE more appropriate than the sigmoid combined with cross-entropy for binary classification problems? What's the intuition behind it?

See the blog post Why You Should Use Cross-Entropy Error Instead Of Classification Error Or Mean Squared Error For Neural Network Classifier Training (2013) by James D. McCaffrey.

It should give you an intuition of why the average cross-entropy (ACE) is more appropriate than MSE (but MSE is also applicable).

In a few words, $\tanh$ + MSE is like sigmoid + MSE, but with labels for classes $-1$ and $1$ instead of $0$ and $1$. If you look at the shape of $\tanh$ function, it has the same flat tails where changes of the argument don't change the result.


Could new training pictures destroy the trained weights of the neural network? Let's say an image has 28*28 pixels, which leads to 784 input nodes in a feed-forward neural network. If an image can be classified into 1 of 10 numbers (e.g. MNIST), there are 10 output nodes.
We train (with gradient descent and back-propagation) the FFNN with a set of known pictures until we get a good accuracy.
Successively, we get a new training picture, which we want to use to train the FFNN even further. However, wouldn't this new training picture destroy the previously learned weights, which have been calibrated to recognize the former training pictures?

Successively, we get a new training picture, which we want to use to train the FFNN even further. However, wouldn't this new training picture destroy the previously learned weights, which have been calibrated to recognize the former training pictures?

This can happen, and happen to various degrees depending on how the neural network is set up, but it is usually something you want to avoid.
Provided a neural network has enough capacity (in terms of number of free parameters), then the function it generates can be flexible enough to approximate the "true" function of the problem you are trying to match to data. Then as long as the learning steps are small enough, adjustments to learn one point on the function will take a larger step towards correct answer on that point than they adjust away from correct answer on other points (intuitively this is about the mutually good solutions being in part orthogonal and/or correlated, so a step towards A does not necessarily move the same distance away from B). Repeating the points again and again will allow the NN to match all the points over time.
Often this is possible to do almost exactly for a large amount of data at the same time. However, this is the opposite problem to your worries in the question, and is considered a bad thing because it can cause overfitting. Overfitting is very easy to achieve with neural networks because they are so flexible and can have large numbers of parameters.
Most often, the goal of training a supervised learner, such as a neural network, is not to exactly match the training data, but to generalise to new data from the same population distribution. You will typically spend far more time and effort combatting overfitting when training a NN, than worrying about lack of capacity to handle multiple data points. Methods to combat overfitting are:

Regularisation. Techniques that adjust a learning algorithm, typically by adding some constraint.
Cross validation. Measuring results of training against unseen data, and picking the best generalised model.


Is a Sobel filter for edge detection a type of Cellular Neural Network? I have implemented a Sobel filter for edge detection in Matlab without using its toolbox. I am a bit confused:

Is a Sobel filter a type of Cellular Neural Network?

Both Sobel and Cellular Neural Network calculate output via its neighborhood cells.

You're right about the basic arrangement of the inputs, but there are a number of differences:

Artificial neural networks typically use exemplar data as inputs for the purpose of training, or adjusting the weights of its internal connections, to accurately classify them within a certain error range. The network is then applied to unknown data to classify them. Edge detection filters are just blind operators that transform input data regardless of how it can be classified. There is no training, so any intelligence exists only in the mind of the filter developer.
A CNN could be trained to be an effective Sobel (edge detection) filter, as described in this paper, but a Sobel filter couldn't be an effective learning algorithm.
Training neural networks is more non-deterministic, with outputs depending on what data they are trained with and potentially even the operational computations that are used for classification. Applying filters is typically deterministic, i.e. they will transform the same data exactly the same way if applied twice.

One succinct way of expressing the biggest difference is: a cellular neural network is looking for a function, while a Sobel filter is a function.

Note that there are types of neural networks called Convolutional Neural Networks, which can use Sobel and other filters in their input layers, as described here. Though, these are neither of the things you are asking about. :)


Is there a continuous conditional variational auto-encoder? The Conditional Variational Autoencoder (CVAE), introduced in the paper Learning Structured Output Representation using Deep Conditional Generative Models (2015), is an extension of Variational Autoencoder (VAE) (2013). In VAEs, we have no control over the data generation process, something problematic if we want to generate some specific data. Say, in MNIST, generate instances of 6.
So far, I have only been able to find CVAEs that can condition to discrete features (classes). Is there a CVAE that allows us to condition to continuous variables, kind of a stochastic predictive model?

Whether a discrete or continuous class, you can model it the same.
Denote the encoder $q$ and the decoder $p$. Recall the variational autoencoder's goal is to minimize the $KL$ divergence between $q$ and $p$'s posterior. i.e. $\min_{\theta, \phi} \ KL(q(z|x;\theta) || p(z|x; \phi))$ where $\theta$ and $\phi$ parameterize the encoder and decoder respectively. To make this tractable this is generally done by using the Evidence Lower Bound (because it has the same minimum) and parametrizing $q$ with some form of reparametrization trick to make sampling differentiable.
Now your goal is to condition the sampling. In other words you are looking for modeling $p(x|z, c;\phi)$ and in turn will once again require $q(z|x, c; \theta)$. Your goal will now intuitively become once again $\min_{\theta, \phi} \ KL(q(z|x, c;\theta) || p(z|x, c; \phi))$. This is still simply transformed into the ELBO for tractability purposes. In other words your loss becomes $E_q[log \ p(x|z,c)] - KL(q(z|x,c)||p(z|c)$.
Takeaway: Conditioning doesn't change much, just embed your context and inject it both into the encoder and decoder, the fact that its continuous doesn't change anything. For implementation details, normally people just project/normalize and concatenate it somehow to some representation of $x$ in both the decoder/encoder.


Machine diagnostics with fuzzy Logic I'm an embedded systems designer, with a little background in fuzzy logic.  I'm developing a device for machine diagnostics based on a series of sensors data.  My question is whether through fuzzy logic I can develop some kind of diagnostic indicator of machine parts failure.

Let's say that I have the Sensor A and sensor B near three parts.  I am wondering if it is possible to get probabilities that part 1, part 2, or part 3 are in failure.

I can collect all physical parameters as temperature, pressure, speed, vibration, etc. related to each of those parts.  Also I have the expert person who can give me info about relation among data collection and failures.  However, I don't have interest in accomplishing this with an expert system.

There is absolutely no doubt that fuzzy logic can contribute to the reliability and accuracy of real time diagnostics of mechanical, thermodynamic, and electro-magnetic devices.  I can state this with assurance because it exists in aeronautics products today.

One of the senior PhDs at the research facility wanted to acquire data via standard techniques and had purchased a brigade of high end Hewlett Packard workstations and instrumentation clusters.  Another senior PhD there wanted to build a fuzzy logic container to provide early warnings of failures, just as implied in the question.  There was some interplay between those two projects in their application in the field.

My direct involvement was to teach the first PhD OO and C++ and design and implement a scientific computing framework to churn the Big Data for the second PhD.  The framework provided a super-set of the union of SciPy and TensorFlow with a little PyTorch added.  That we developed these techniques before Java and Python existed is why Hewlett Packard sent engineers to study what we were doing, with DoD permission, of course.

What you want to do is this:

Become familiar with the open source options in Java, Python, and C.  The general trend is to either use Java-ML or use C wrapped in the above mentioned Py libraries.
Use 16 or 24 bit multi-channel data acquisition with time stamps and latched, concurrent A-to-D conversion at a frequency at least twice1 the 16th harmonic of the vibration base frequencies.
Ensure that you have sufficient system memory to hold what for embedded qualifies as Big Data2.
Ensure that you know how spectra (both amplitude and phase) can be obtained via FFT from channels.
Use LINUX, a single board computer, and either Intel Movidius3 or NVidia CUDA for hardware acceleration.
The fuzzy logic container and any assistive reinforced learning can be on the embedded CPU and leverage the hardware acceleration.
The fuzzy rules can be acquired from the experts to which you have access, as indicated in the question.
Read Chaos Theory Tamed by Williams, 1997 to get familiar with phase relationships between input channels and how to analyze changes in chaotic behavior between the harmonics.

Start with small POCs and generate independent components that can be connected and tested independently with input that came from the output of previously tested components.  Here are some, in rough order of development and signal path.

Data acquisition of multiple samples of multiple channels to a matrix.
FFT of acquired data to an amplitude-phase array in the frequency domain.
Analysis to determine the patterns in metrics that precede failure.
Introduce and train artificial networks (either LSTM or attention type) where indicated to remove redundancy from the information that will be input to the fuzzy logic container.
Write up the rules and place them in the container.
Train the fuzz.
Test the system.

You will need to work with the experts on most of these steps.  If you try to involve them at the rule writing stage and they weren't privy to any of the previous developments, they may be confused.

I'm not a lawyer, but, because of the involvement of failure prediction experts, I think you'll need an internationally binding NDA to ensure the experts don't walk away with the product of your research and move to another country.

These are a few considerations for product reliability you may need to consider even after you have a working system.

Develop a way to adapt the system in real time ... to tracking changes in what indicates near failure based on a centralized database related to the product.
Consider version control and deployment challenges.

I'm sorry that I have no further references.  You can try the IEEE library if you have access.  I have been too busy doing this type of work to publish what we do, but I think most of the important directions are included here already.

FootNotes

[1] Not much can be learned about impending failure from the primary harmonic of vibration.  The first 16 harmonics are sufficient.  Because of Nyquist Criteria considerations, the sample rate must be twice the top harmonic.  This means that if your vibration is 1 KHz, the sample rate must be at least 32 KHz.

[2] Big Data input rates may be necessary because with 8 channels of strain input from Wheatstone bridges of gauges, 8 channels of accelerometer, 8 channels of IR sensing and the above vibration base harmonic, we have (24 channels + 1 time stamp) x 32 KHz x 3 bytes per channel-sample = 24 Mbytes per second.  If your failure detection window holds 10 seconds of vibration data as IEEE 64 bit reals and you send 1 second (overlapping) Hamming windows to the hardware FFT, just for that alone will need 12.8 Gbyte input matrix, 6.4 for the input and 6.4 for the output.

[3] Available for embedded applications from Mouser and other distributors as properly paired single board computers and Movidius mPCIe (mini PCI express) daughter boards from makers like Aaeon.


What are AI use cases for communication service providers? I am currently looking for AI use cases for Telco. What are the different AI use cases for Telcos/communication service providers?

AI could hold the key in automating and optimizing networks. On the subscriber side, ML and AI will assist telecom operators in profiling the subscribers. This will be achieved by analyzing network activity, conversion rate of offers and data usage trends.

Below are a few use cases and how they will transform the telecommunication sector. (Source H2o.ai blog https://www.h2o.ai/telecom/ )

Old generation telecom technologies.

Reactive Maintenance
Network optimization with human intervention
Centralized intelligence
Security attack repair
Backlogged customer tickets

Future generation AI based telecom technologies.

Predictive Maintenance
Self-optimizing network
Optimal network quality
Intelligence at the edge
Security attack prediction
Improved customer experience through customer service chat bots.
Speech and voice services for customer which allows users to explore media content by spoken word rather than remote control.
Predictive maintenance which is the ability to fix problems with telecom hardware such as cell towers, power lines e.t.c before they happen by detecting signals that usually lead to failure.


When should we use algorithms like Adam as opposed to SGD? As far as I know, Stochastic Gradient Descent is an optimization algorithm which belongs to the the category of algorithms where hyper-parameters have to be defined beforehand. They are useful in many cases, but there are some cases that the adaptive learning algorithms (like AdaGrad or Adam) might be preferable.

When are algorithms like Adam and AdaGrad preferred over SGD? What are the cons and pros of adaptive algorithms, like Adam, when we compare them with learning algorithms like SGD?

Empirically, I observed that algorithms like Adam and RMSProp tended to give me a final higher performance (in my case, the accuracy) on (the validation dataset) with respect to SGD. However, I also observed that Adam and RMSProp are highly sensitive to certain values of the learning rate (and, sometimes, other hyper-parameters like the batch size) and they can catastrophically fail to converge if e.g. the learning rate is too high. On the other hand, in general, SGD have not led me to the highest performance, but they did not catastrophically fail (at least, as much as Adam and RMSProp) in my experiments (even when using quite different hyper-parameters). I noticed that the learning rate (and the batch size) are the hyper-parameters that mainly affect the performance of all these algorithms.

In my experiments, I use SGD without momentum and I used the (PyTorch) default values of Adam and RMSProp. I only compared SGD with Adam and RMSProp, on the relatively simple task of recognising MNIST digits. You can have a look at this repository https://github.com/nbro/comparative-study-between-optimizers, which contains the code I used to perform these experiments. You also have the instructions there to perform the experiments (if you want).


What is the purpose of the noise injection in the generator network of a GAN? I do not understand why with enough training how the generator cannot learn all images from the training set as a mapping from the latent space - It is the absolute optimal case in training as it replicates the distribution and the discriminator output will always be 0.5. Even though most blog posts I have seen do not mention noise, a few of them have them in their diagrams or describe their presence, but never exactly describe the purpose of this noise.

Is this noise injected to avoid the exact reproduction of the training data? If not what is the purpose of this injection and how is exact reproduction avoided?

Your goal is to model a distribution when constructing a GAN, therefore you need a way to be able to sample that distribution. The noise's purpose is so you can do this. Generally, it's drawn from a distribution that is computationally easy to draw from (like a gaussian).

You are modeling the generator $G(X)$  where $X \sim N(\mu, \sigma^2)$. this means $G(X)$ is a random variable itself. The forward pass of the network transforms the $X$ samples into our $G(X)$ samples, allowing us to formulate a loss function (by solving the expectation as the mean of drawn samples) and train the model.

Takeaway: The noise injected is just a parametrization of our Generator in another space, and the training goal is to learn the ideal transformation (we use neural networks because they are differentiable and are effective function approximators)

Also, note to your point of why it doesn't learn the training data (in its entirety) exactly is because generally $G(X)$ is continuous, and therefore if it has 2 images in its codomain, there also exists some path in pixel space from one to the other containing an uncountable (or quantized countable) number of images that don't exist in the training, and this would be reflected in the loss, therefore in the min-max game of the optimization, its difficult for it learn the training set on the nose.


How can neural networks deal with varying input sizes? As far as I can tell, neural networks have a fixed number of neurons in the input layer.

If neural networks are used in a context like NLP, sentences or blocks of text of varying sizes are fed to a network.
How is the varying input size reconciled with the fixed size of the input layer of the network? In other words, how is such a network made flexible enough to deal with an input that might be anywhere from one word to multiple pages of text?

If my assumption of a fixed number of input neurons is wrong and new input neurons are added to/removed from the network to match the input size I don't see how these can ever be trained.

I give the example of NLP, but lots of problems have an inherently unpredictable input size. I'm interested in the general approach for dealing with this.

For images, it's clear you can up/downsample to a fixed size, but, for text, this seems to be an impossible approach since adding/removing text changes the meaning of the original input.

Three possibilities come to mind.

The easiest is the zero-padding. Basically, you take a rather big input size and just add zeroes if your concrete input is too small. Of course, this is pretty limited and certainly not useful if your input ranges from a few words to full texts.

Recurrent NNs (RNN) are a very natural NN to choose if you have texts of varying size as input. You input words as word vectors (or embeddings) just one after another and the internal state of the RNN is supposed to encode the meaning of the full string of words. This is one of the earlier papers.

Another possibility is using recursive NNs. This is basically a form of preprocessing in which a text is recursively reduced to a smaller number of word vectors until only one is left - your input, which is supposed to encode the whole text. This makes a lot of sense from a linguistic point of view if your input consists of sentences (which can vary a lot in size), because sentences are structured recursively. For example, the word vector for "the man", should be similar to the word vector for "the man who mistook his wife for a hat", because noun phrases act like nouns, etc. Often, you can use linguistic information to guide your recursion on the sentence. If you want to go way beyond the Wikipedia article, this is probably a good start.


Is there a shortage of machine learning and artificial intelligence professionals? I have heard that there is a huge talent demand in AI, with limited talent supply. Is it true?

In one word – yes, there is!

If you are following the industry reports then you must know that more than 500,000 jobs are lying vacant because there is a huge shortage of qualified talent in the industry.  It is also quite interesting to note that the market has almost double the number of jobs available than the required talented ML and AI professionals – just in case – there is someone out there.

Another thing to note is that the countries like US, China, India, Israel, Germany, Switzerland, Canada, France, Spain and Singapore —  have seen a consistent rise in demand for skilled ML and AI professionals.


How are LSTM's trained for text generation? I've seen some articles about text generation using LSTMs (or GRUs) for text generation.

Basically it seems you train them by folding them out, and putting a letter in each input. But say you trained it with text which includes the string:

"The dog chased the cat"

and also includes the string:

"The lion chased the ca"

Both should be acceptable. But although they are very similar they differ entirely after the 4th character. So they will result in two very different vectors. Further the longer you roll it out the more they will differ.
How is it then possible for an LSTM to learn that "The [something] chased the cat" is an acceptable phrase?

Equally if you train it to try and learn to pair up parentheses. I can see how you could manually design it to do this, but how exactly could it be trained to do this just by entering strings like "(das asdasdas) axd"?

What I'm getting at is that I don't get how it could LEARN any sort of structure more than a Markov model.

Any ideas?

(Also, I've only ever seen one article that showed a LSTM that can pair parentheses. So has this study ever been replicated?) I get that it is possible for an LSTM to do this but I don't get how it can learn to do this!

What I'm getting at is you usually train things with the input phrase and then compare it to the expected phrase to get the error. But in text generation there might be millions of possible acceptable expected phrases! So how can you compute an error?

You compute the error in the same way as normal - treat the actual output as "ground truth" even if other options are possible. The LSTM will output probabilities of next character, it will not learn to associate a single "true" output except for rare circumstances (such as completing a word that can only be done in one way according to the training data). In practice that means that understanding acceptable loss values in NLP training is different to understanding them in other supervised learning environments, typically being higher than for datasets where ground truth is more predictable from the data.

In this way, it is a lot like a Markov or n-gram model, except that it learns its own internal state representations (they are not prescribed by the model, but are limited by structure and data available). The gated RNN memory models like LSTM and GRU include a mechanism for handling long term memory so can relatively easily learn things like quote, paren pairs etc.

  What I'm getting at is that I don't get how it could LEARN any sort of structure more than a Markov model.

A Markov model can in theory represent any sequence knowledge that an LSTM can. The difference is in terms of efficiency. RNNs have more compact internal representations than explicit state representation in say HMMs. This makes them harder to work with in some ways (it is hard to see what a RNN state vector represents), but means that they can store more complex knowledge about state transitions in less memory. It also means that RNNs can generalise (although not guaranteed correctly) more readily, they will not fall back to random behaviour given a brand new state without any training history.


how to benefit from previous training weights in training again to increase accuracy? I have trained  a modified VGG classification CNN, with random initialized weights; therefor the validation accuracy was not high enough for me to accept (around 66%).
now using the weights resulted from training the network, how can i use those weights in training the network again to improve accuracy? (e.g. using previous training weights with different learning rate, or increase epochs, ..)

First, I assume you've tuned your hyperparameters. Because, instead of re-train the network (use the weights that resulted from the previously training process) that needs more times, I'll invest more on hyperparameters tuning of the available network.

Then, there are several methods and considerations:

You can use the weight resulted from your first network as the initial of your next training process. But as the network will face the same data/problem if you use "initial" value for your hyperparameters (e.g. high learning rate) then I'm afraid it will lead to overfitting. Because it's simply "nothing change" in your architecture.
If you previously didn't use adaptive learning for your training process (e.g. Adadelta, Adam), you can re-train your network with a smaller learning rate. So, your model can find a better result.
Or, you can use the concept of the selffer network, you use the weight for some layers from previous training process (you can freeze it or not) and randomly initialize other layers. And then, train the network using "initial" value of hyperparameters

You can read more about the transfer learning concept (or selffer network) to get the most appropriate method for your case. There is also a paper about "incremental training on CNN" that I think is similar with the selffer network but with some modifications.

Hope it helps


How can people contribute to AGI research? Is there a way for people outside of the core research community of AGI to contribute to the cause?

There are a lot of people interested in supporting the field, but there is no clear way to do that. Is there something like BOINC for AGI researches, or open projects where random experts can provide some input? Maybe Kickstarter for AGI projects?

OpenCog is an open source AGI project. But it is is also incredibly complex and IMHO not a good idea (I have not fully read his theories). You can learn the essential ideas behind OpenCog from the co-founder Ben Goertzel site as well.

Or, you can participate in the philosophical discussion regarding AGI. For strictly AGI, decision theory, logic, and math material (they are all related), you can look up stuff from http://yudkowsky.net/ or https://arbital.com/. But, in some sense, every branch of philosophical inquiry can be tied back to AGI and consciousness (ethics, metaphysics, etc.), so if it fancies you it depends on how you'd like to tackle it.

You could also study the psychology end of things. The following papers and related ideas are quite important in the field of study of consciousness and cognition (but keep in mind this is pretty much a random list, the literature is massive!):

Global Workspace: A Theory of Consciousness (1997)
Relevance Realization and the Emerging Framework in Cognitive Science (2009)
The free-energy principle: a unified brain theory? (2010)

Recently, (in mathematical time), progress in category theory shows promise of being a unified framework of much of the existing math. I know next to nothing about it, but the people that do are applying to many new fields of study (including AI, apparently). Category theory requires a lot of background mathematical knowledge before its vocabulary begins to make sense, though, so beware. You can read about it on the nCatLab and occasionally on John Baez's blog: Azimuth

Of course, "regular" techniques in Machine Learning such as neural networks, reinforcement learning, statistical methods, and others are very powerful as well, but due to certain regards in their construction, they are generally understood as only being capable of being "narrow AI" in the sense that they can only complete a single task very well, but perhaps you can find some research that changes this?


Implementing SARSA for a 2-stage Markov Decision Process I am a bit confused as to how exactly I should be implementing SARSA (or Q-learning too) on what is a simple 2-stage Markov Decision Task. The structure of the task is as follows:

Basically, there are three states $\{S_1,S_2,S_3\}$ with $S_1$ is in the first stage for which the two possible actions are the two yellow airplanes. $S_2$ and $S_3$ are the possible states for the second stage and the feasible actions are the blue and red background pictures, respectively. There is only a reward at the end of the second stage choice. If I call the two first stage actions $\{a_{11},a_{12}\}$ and the four possible second stage actions $\{a_{21},a_{22},a_{23},a_{24}\}$, from left to right, then a sample trial/episode will look like:
$$S_1, a_{11}, S_2, a_{22},R \quad \text{ or }\quad S_1, a_{11}, S_3, a_{24}, R.$$
In the paper I am reading, where the figure is from, they used a complicated version of TD$(\lambda)$ in which they maintained two action-value functions $Q_1$ and $Q_2$ for each stages. On the other hand, I am trying to implement a simple SARSA update for each episode $t$:
$$Q_{t+1}(s,a)= Q_t(s,a) + \alpha\left(r + \gamma\cdot Q_t(s',a') - Q_t(s,a)\right).$$
In the first-stage, there is no reward so an actual realization will look like:
$$Q_{t+1}(S_1, a_{11}) = Q_t(S_1,a_{11})+\alpha\left( \gamma\cdot Q_t(S_3,a_{23}) - Q_t(S_1,a_{11})\right).$$
I guess my confusion is then how should it look like for the second stage of an episode? That is, if we continue the above realization of the task above, $S_1, a_{11}, S_3, a_{23}, R$, would should fill in the $?$:
$$Q_{t+1}(S_3,a_{23}) = Q_t(S_3,a_{23}) + \alpha\left(R +\gamma\cdot Q_t(\cdot,\cdot)-Q_t(s_3,a_{23}) \right)$$
One on hand, it seems to me that since this is the end of an episode, we assign $0$ to the $Q_t(\cdot,\cdot).$ On the other hand, the nature of this task is that it repeats the same episode over time for a total of $T$, a large number, times we need $Q_t(\cdot,\cdot) = Q_t(S_1,\cdot),$ with the additional action-selection in the first stage there.
I will greatly appreciate if someone can tell me what is the right way to go here.
The link to paper

In this game you can view end of an episode two ways:

There is an implied, terminal, fourth state $s_4$ representing the end of the game.

You could view the process as a continuous repeating one, where no matter what the choice is made in $s_2$ or $s_3$, the following state is $s_1$.

The first, terminating, view is a simpler and entirely natural view since nothing that the agent does in one episode can influence the next. It will result in a Q table that predicts future rewards within a single episode for the current agent (as opposed to discounted view over multiple episodes).
You are over-complicating things for yourself by ignoring that a zero reward is still a reward (of $0$). There is no need to remove $R$ from your initial update rule. In many environments there are rewards collected before the end of an episode.
In addition, to complete the standard episodic view, you can note that $Q(s_4, \cdot) = 0$ always by definition, hence so does $\text{max}_{a'}[Q(s_4, a'] = 0$. It is common here though to have a branch based on detecting a terminal state, and use a different update rule:
$$Q_{t+1}(S_3,a_{23}) = Q_t(S_3,a_{23}) + \alpha\left(R - Q_t(s_3,a_{23}) \right)$$
In brief, most implementations of TD algorithms do this:

Always assume a reward on each time step, which can be set to $0$

Special case for end of episode with a simplified update rule, to avoid needing to store, look up or calculate the $0$ value associated with terminal states

When implementing the environment, it is common to have a step function that always returns reward, next state and whether or not it is terminal e.g.
reward, next_state, done = env.step(action)

Details may vary around this. If you are working with an environment that does not have such a function (many will not have an inherent reward), then it is common to implement a similar function as a convenient wrapper to the environment so that the agent code does not have to include calculations of what the reward should be or whether the state is terminal.


Are Relational DBs and SQL used in Expert Systems? In the book Prolog Programming for Artificial Intelligence, a large and intricate chapter (chapter 14) is dedicated to Expert Systems. In these systems, a  knowledge-database is represented through facts and rules in a declarative manner, and then we use the PROLOG inference engine to derive statements and decisions.
I was wondering: are there any examples of expert systems that represent knowledge through a standard Relational Database approach and then extract facts through SQL queries? Is there any research in this area? If not, why is a rule-based approach preferred?

A recent research example is the "Grind" system. Take a look at the paper Computing FO-Rewritings in $\mathcal{E} \mathcal{L}$ in Practice: from Atomic to Conjunctive Queries (2018) by Peter Hansen and Carsten Lutz. Here's the abstract.

A prominent approach to implementing ontology-mediated queries (OMQs) is to rewrite into a first-order query, which is then executed using a conventional SQL database system. We consider the case where the ontology is formulated in the description logic $\mathcal{E} \mathcal{L}$ and the actual query is a conjunctive query and show that rewritings of such OMQs can be efficiently computed in practice, in a sound and complete way. Our approach combines a reduction with a decomposed backwards chaining algorithm for OMQs that are based on the simpler atomic queries, also illuminating the relationship between first-order rewritings of OMQs based on conjunctive and on atomic queries. Experiments with real-world ontologies show promising results.


What is the effect of using pooling layers in CNNs? I know how pooling works, and what effect it has on the input dimensions - but I'm not sure why it's done in the first place. It'd be great if someone could provide some intuition behind it - while explaining the following excerpt from a blog:

  A problem with the output feature maps is that they are sensitive to the location of the features in the input. One approach to address this sensitivity is to down sample the feature maps. This has the effect of making the resulting down sampled feature maps more robust to changes in the position of the feature in the image, referred to by the technical phrase “local translation invariance.”

What's local translation invariance here?

Pooling has multiple benefits

Robust feature detection.
Makes it computationally feasible to have deeper CNNs

Robust Feature Detection

Think of max-pooling (most popular) for understanding this.
 Consider a 2*2 box/unit in one layer which is mapped to only 1 box/unit in the next layer (Basically pooling). Let's say the feature map (kernel) detects a petal of a flower. Then qualifying a petal if any of the 4 units of the previous layer is fired makes the detection robust to noise. There is no strict requirement that all 4 units should be fired to detect a petal. Thus, the next layer (after pooling) captures the features with noise invariance. We can also say it is local translation invariance (in a close spatial sense) as a shifted feature will also be captured. But also remember translation invariance in general is captured by the convolution with kernels in the first place. (See how 1 kernel is convolved with the whole image)

Computational advantage

The dimensions of the inputs in image classification are so huge that the number of the multiplication operation is in billions even with very few layers. Pooling the output layer reduces the input dimension for the next layer thus saving computation. But also now one can aim for really deep networks(number of layers) with the same complexity as before.


What is the purpose of the hidden layers? Why would anybody want to use "hidden layers"? How do they enhance the learning ability of the network in comparison to the network which doesn't have them (linear models)?

"Hidden" layers really aren't all that special... a hidden layer is really no more than any layer that isn't input or output. So even a very simple 3 layer NN has 1 hidden layer. So I think the question isn't really "How do hidden layers help?" as much as "Why are deeper networks better?".

And the answer to that latter question is an area of active research. Even top experts like Geoffrey Hinton and Andrew Ng will freely admit that we don't really understand why deep neural networks work. That is, we don't understand them in complete detail anyway.

That said, the theory, as I understand it goes something like this...  successive layers of the network learn successively more sophisticated features, which build on the features from preceding layers. So, for example, an NN used for facial recognition might work like this: the first layer detects edges and nothing else. The next layer up recognizes geometric shapes (boxes, circles, etc.). The next layer up recognizes primitive features of a face, like eyes, noses, jaw, etc. The next layer up then recognizes composites based on combinations of "eye" features, "nose" features, and so on.

So, in theory, deeper networks (more hidden layers) are better in that they develop a more granular/detailed representation of a "thing" being recognized.


What is the "thing" which is trained in AI model training I am a newbie in the fantastic AI world, I have started my learning recently.
After a while, my understanding is, we need to feed in tremendous data to train a or many models.

Once the training is complete, we could take out the trained models and "plug in" to any other programming languages to use to detect things.

So my questions are:

1. What are the trained models? are they algorithms or a collection of parameters in a file?

2. What do they look like? e.g. file extensions

3. Especially, I want to find the trained models for detecting birds (the bird types do not matter). Are there any platforms for open-source/free online trained AI models??

Thank you!

This answer applies to Machine Learning (ML) part of AI, as that seems to be what you are asking about. Please bear in mind that AI is still a broad church, including many other techniques than ML. ML, including neural networks for deep learning, and Reinforcement Learning (RL) is only a subset of AI - some AI techniques are more focused on the algorithm than parameters.

  What are the trained models? are they algorithms or a collection of parameters in a file?

In ML, the usual process is to feed data into parametric function (e.g. a neural network) and alter the parameters of it to "fit" the data. The main output of this is a collection of parameters and hyperparameters that describe the parametric function. So 90% of the time, when discussing the "trained model", it means the same thing as the collection of paramaters.

However, those parameters are of limited use without a library that can re-create the function from them. Parameters will be saved from a specific library and can be loaded back into that library easily. It is also possible for libraries to read or convert from models saved from other libraries, much like how different spreadsheet programs can read each others' files.

  What do they look like? e.g. file extensions

This varies a lot, depending on which library was used. It is not possible to make a general statement. For example, Tensorflow can save variables to a "Checkpoint" file with .ckpt extension, but can get more sophisticated depending on how much of the model you want to export, and full models with whole structure will contain more than just the variables and have the .pb extension.

  Especially, I want to find the trained models for detecting birds (the bird types do not matter). Are there any platforms for open-source/free online trained AI models??

There are a few places where you can find selections of pre-trained models. One such place is Tensorflow's Model Zoo and you might be interested in Tensorflow detection model zoo.

Other frameworks may also provide example code. For instance Caffe also has a "Model Zoo" (searching Model Zoo is a good starting strategy).

If you are working at the level of collecting model parameters and want to run these models yourself, you will need to learn a bit about each library, what language is used to work with it, and maybe follow some tutorial about how to use it. A few models will be packaged up with working scripts to use from the command line, but many are not and may take sometime and effort to get working.

When you have a specific detection target, you may be disappointed to find models that don't quite match what you want. For image classifying, it is common if you have a specialist need to take a pre-existing general model that has been trained on large dataset for weeks, and then "fine tune" it with your own image dataset for your purpose. Most NN libraries will have tutorials and examples of this fine tuning process.


Will artificial intelligence make the human more rational? With so much innovation, with so much previous human manual labor being performed in minutes or seconds by an artificial intelligence, one day man will put the survival and propagation of his species above his ideologies and cultures.

I am worried because we are living the fourth industrial revolution, and this will generate millions of unemployment, even if new jobs are created in the future. The problem is that a lot of humans worry about their own job, and not about their own children's future. This is completely retrograde.

Will, one day, Artificial Intelligence be able to direct us towards an intelligent path as a propagation of the species, or else center the focus of humanity on something that it adds?

I would counter this question with another one:
Is rationality all we strive for?

Intelligence is a waste ground, and we have not even defined which intelligence we're talking about: Emotional Intelligence (empathy), Rational/Intellectual Intelligence (IQ), artistic intelligence, etc. etc. etc.

Something which demarks us as humans is that sometimes we're able to deal with to counteracting intelligences. Logic sometimes dictates one thing, but logic is bound to what parameters are taken into account, and mostly it's about quantifying. an intelligence which does qualitative logic behaves sometimes different, and suggests sometimes other actions.
As humans we weight and counterweight these factors.

Politics is a good example. In most countries there are 2 or 3 rational approaches to problem solving (to make it easy let's call them left, center, right - though I know in fact it's more positions and in an at least two dimensional space, not linear).
But what I mean is all those positions are rational, nevertheless it's other types of intelligence who define their the way they use their ratio.

So in the end this fourth step is just a continuity and I assume that with time theoretically at least the shift should be away from blue collar jobs to more white collar jobs until there are no blue collar jobs.
So far the theory I guess, but then again? Would this be smart? Or intelligent? A world where machines make all the dirty work and we're just supervisors?

My thought to this is we will still need mechanics, probably with more technology always more, just that their focus will shift. 20 years ago mechanics were teached analogue technology, nowadays they learn more digital technology (best example would be car mechanics), but there you see: They're still needed, we also need people with analogue knowledge to mantain older systems which cannot or will not be converted to digital era. And I think that's OK!
So from my point of view, nobody has to fear to loose it's job, but instead embrace, that when you keep your knowledge up to date, there will always be a job for you.
Nowadays for example IT-professionals who are able to code older computer languages are better paid than ever, because their scarce...


Accuracy too high too fast? I have a simple text classifier, with the following structure:

    input = keras.layers.Input(shape=(len(train_x[0]),))

    x=keras.layers.Dense(500, activation='relu')(input)
    x=keras.layers.Dropout(0.5)(x)
    x=keras.layers.Dense(250, activation='relu')(x)
    x=keras.layers.Dropout(0.5)(x)
    preds = keras.layers.Dense(len(train_y[0]), activation="sigmoid")(x)

    model = keras.Model(input, preds)

When training it with 300,000 samples, with a batch size of 500, I get an accuracy value of .95 and loss of .22 in the first iteration, and the subsequent iterations are .96 and .11.

Why does the accuracy grow so quickly, and then just stop growing?

It actually depends on a couple of things here -

How many output classes do you have? If you have only 2 or 3 classes, it is a very easy task for the classifier that you have built. So, it is highly possible that convergence has occurred.
As @Djib2011 mentioned already, if your input training set is not balanced and is heavier with one of the output classes (95%), then this accuracy you see makes sense but note that your model won't do well in production.
Do not try to evaluate your model on the basis of your training accuracy. Test it on data your model has never seen before and then evaluate the classification accuracy making sure that your training/testing data is not heavy with one of the classes.


Does this tutorial use normalization the right way? There is this video on pythonprogramming.net that trains a network on the MNIST handwriting dataset. At ~9:15, the author explains that the data should be normalized.

The normalization is done with

x_train = tf.keras.utils.normalize(x_train, axis=1)
x_test = tf.keras.utils.normalize(x_test, axis=1)

The explanation is that values in a range of 0 ... 1 make it easier for a network to learn. That might make sense, if we consider sigmoid functions, which would otherwise map almost all values to 1.

I could also understand that we want black to be pure black, so we want to adjust any offset in black values. Also, we want white to be pure white and potentially stretch the data to reach the upper limit.

However, I think the kind of normalization applied in this case is incorrect. The image before was:

After the normalization it is

As we can see, some pixels which were black before have become grey now. Columns with few black pixels before result in black pixels. Columns with many black pixels before result in lighter grey pixels.

This can be confirmed by applying the normalization on a different axis:

Now, rows with few black pixels before result in black pixels. Rows with many black pixels result in lighter grey pixels.

Is normalization used the right way in this tutorial? If so, why? If not, would my normalization be correct?

What I expected was a per pixel mapping from e.g. [3 ... 253] (RGB values) to [0.0 ... 1.0]. In Python code, I think this should do:

import numpy as np
import imageio
image = imageio.imread("sample.png")
image = (image - np.min(image))/np.ptp(image)

You could rank every pixel in terms of brightness before normalization and after to verify that the ranks are preserved. Correct normalization would preserve the  ranks. From the pictures 1 and 2, it seems that the ranks were not preserved (e.g. the top right pixels went from grey to black).
The tutorial's normalization is done incorrectly: x_train is a an array of 2D images, but the normalization was applied along the first axis. What happened is that each column of every image was normalized relatively to itself (compare image 1 and 2 column-wise – the ranks were preserved). The normalization should have been applied along the second axis – that would normalize every pixel relative to the image it's in. In seems that incorrect normalization wasn't enough to sabotage the learning though!
Your proposed normalization should work fine.


Which approaches could I use to create a simple chatbot using a neural network? I wanted to start experimenting with neural networks, so I decided to make a chatbot (like Cleverbot, which is not that clever anyway) using them.
I looked around for some documentation and I found many tutorials on general tasks, but few on this specific topic. The one I found just exposed the results without giving insights on the implementation. The ones that did, did it pretty shallowly (the TensorFlow documentation page on seq2seq is lacking, IMHO).
Now, I feel I may have understood the principle more or less, but I'm not sure and I am not even sure how to start. Thus I will explain how I would tackle the problem and I'd like feedback on this solution, telling me where I'm mistaken, and possibly have any link to detailed explanations and practical knowledge on the process.

The dataset I will use for the task is the dump of all my Facebook and WhatsApp chat history. I don't know how big it will be but possibly still not large enough. The target language is not English, therefore I don't know where to quickly gather meaningful conversation samples.

I am going to generate a thought vector out of each sentence. Still don't know how, actually; I found a nice example for word2vec on the deeplearning4j website, but none for sentences. I understood how word vectors are built and why, but I could not find an exhaustive explanation for sentence vectors.

Using thought vectors as input and output I am going to train the neural network. I don't know how many layers it should have, and which ones have to be LSTM layers.

Then there should be another neural network that is able to transform a thought vector into a sequence of characters composing a sentence. I read that I should use padding to make up for different sentence lengths, but I miss how to encode characters (are codepoints enough?).

I would recommend to start by reading this blogpost.
You can probably cannibalise the code to create a RNN that takes in one statement of a dialogue and then proceeds to output the answer to that statement.

That would be the easy version of your project, all without word vectors and thought vectors. You are just inputting characters, so typos don't need to concern you.

The next more complex step would be to input word vectors instead of characters. That would allow you to generalise to words that aren't part of your training data. And it is probably still just a minor modification of the code.

If you insist on using thought vectors, you should start reading up on NN translation. And probably try to get a pre-trained encoder network. Or pre-train it yourself on a large translation corpus for your language.

With your small training set the best you can do is probably massively overfit until your system recreates your training data verbatim. Using word vectors will allow your system to give the same answer to "I beat the cat today." as you gave in the training data to "I kicked the dog yesterday."

I'm not sure thought vectors will make a big difference. If you get the decoder to learn at all.