Why don't integrated gradients explain samples correctly? I have a linear tabular dataset made of floats. The dataset follows a simple rule like:
if features A and B are in a certain range then target class is 1, otherwise target class is 0.

Since I want to get some interpretability from my ANN model, I opted for using the integrated gradients method implemented by alibi.
Unfortunately, most of individual samples don't show A and B as the leading features as expected. Even more weird is the fact that, when I average the attributions of all the individual samples, A and B get the highest score. In other words, local explanations fail but, on average, the global explanation is correct.
Can anyone help me out to understand why this happens? Isn't integrated gradients method suitable for tabular datasets?
By the way, my baseline is based on a uniform distribution of random floats ranging from 0 to the maximum of each column.

After several tests, I've opted for the following configuration:
Approximation mode: riemann trapezoidal
Steps: 1000
Baseline: average mean of each column (feature)
Although the global score remains good, there're important variations in the number of correctly explained local samples. For example, an execution returns 200 out of 390 samples explained as expected, but the following execution only 17.
Is there a reason for that? And, is there a way to make the algorithm predictable?


What are the real applications of hierarchical temporal memory? What are the real applications of hierarchical temporal memory (HTM) in machine learning (ML) these days?

The following figure [1] is inspiring:

It means HTM can be used as a memory unit in different neural networks such as RNN. In the same reference, we can find the following advantages and applications of HTM:

  Sequence learning:
  Being able to model temporally correlated patterns represents a key property of intelligence, as it gives both biological and artificial systems the essential ability to predict the future. It answers the basic question “what will happen next?” based on what it has seen before. Every machine learning algorithm should be able to provide valuable predictions not just based on static spatial information but also grounding it in time.
  High-order predictions:
  Real-world sequences contain contextual dependencies that span multiple time steps, hence the ability to make high-order predictions becomes fundamental. The term “order” refers to Markov's order, specifically the minimum number of previous time steps the algorithm needs to consider in order to make accurate predictions. An ideal algorithm should learn the order automatically and efficiently.
  Multiple simultaneous predictions:
  For a given temporal context, there could be multiple possible future outcomes. With real-world data, it is often insufficient to only consider the single best prediction when information is ambiguous. A good sequence learning algorithm should be able to make multiple predictions simultaneously and evaluate the likelihood of each prediction online. This requires the algorithm to output a distribution of possible future outcomes.
  Continual Learning:
  Continuous data streams often have changing statistics. As a result, the algorithm needs to continuously learn from the data streams and rapidly adapt to changes. This property is important for processing continuous real-time perceptual streams, but has not been well studied in machine learning, especially without storing and reprocessing previously encountered data.
  Online learning:
  For real-time data streams, it is much more valuable if the algorithm can predict and learn new patterns on-the-fly without the need to store entire sequences or batch several sequences together as it normally happens when training gradient-based recurrent neural networks. The ideal sequence learning algorithm should be able to learn from one pattern at a time to improve efficiency and response time as the natural stream unfolds.
  Noise robustness and fault tolerance:
  Real world sequence learning deals with noisy data sources where sensor noise, data transmission errors and inherent device limitations frequently result in inaccurate or missing data. A good sequence learning algorithm should exhibit robustness to noise in the inputs.
  No hyperparameter tuning:
  Learning in the cortex is extremely robust for a wide range of problems. In contrast, most machine-learning algorithms require optimizing a set of hyperparameters for each task. It typically involves searching through a manually specified subset of the hyperparameter space, guided by performance metrics on a cross-validation dataset. Hyperparameter tuning presents a major challenge for applications that require a high degree of automation, like data stream mining. An ideal algorithm should have acceptable performance on a wide range of problems without any task-specific hyperparameter tuning.

One of the applications of HTM is in Anomaly detection. See, for example, this paper Unsupervised real-time anomaly detection for streaming data (2017) which used from a network of HTMs to solve this problem in streaming data.

The implementation of HTM (NuPIC) can help to see more applications of HTM in ML.


Comparing a large/general CNN to a smaller more specialized one? I am still somewhat a novice in the ML world, but I had a strange idea about CNNs and wanted to ask if this would be a valid way to check the robustness of a general CNN that classifies certain images.
Let's say that I make a CNN that takes in many different images of sports players performing a certain action (basketball shot, football kick, freestyle in swimming, flip in gymnastics, etc). Firstly, would it be possible for such a CNN to distinguish between such varied images and classify them accurately? And if so, can it be a good idea to compare this "larger" CNN to multiple "smaller" more specialized ones that take in images from one particular sport?
In other words, I want to know that if I have a "larger" CNN that gives me an output like "football being kicked", is there a way to then double-check that output with a smaller CNN that only focuses on football moves? In essence, could we create a  system where once you obtain an output from a general CNN, it automatically classifies the same image through a more specialized CNN, and then if the results are of similar accuracy, you know for sure that CNN works?
Kind of like having a smaller CNN as a "ground-truth" for the bigger one? In my head it kind of goes like this:
large_net_output = 'Football kick identified with 95.56% confidence'

for sport in large_net:
    if sport == 'football':
        access = small_net_for_football
        return small_net_for_football_output

    elif sport == 'swimming':
        access = small_net_for_swimming
        return small_net_for_swimming_output

    elif sport == 'baseball':
        access = small_net_for_baseball
        return small_net_for_baseball_output

# and so on....
>>> small_net_for_football_output = 'Football kick identified with 97.32% confidence'

robustness_check = large_net_output - small_net_for_football_output
print(robustness_check)

>>> 'Your system is accurate within a good range of 1.76%'

I hope this makes sense, and that this question does not cause any of your to cringe. Would appreciate any feedback on this!

After reading your question I can relate it to the Representation Learning papers such as SimCLR and SwAV. These models use a "Big Task agnostic CNN" to obtain smaller representations of the images and then they train another CNN for classification. I suggest you read Big Self-Supervised Models are Strong Semi-Supervised Learners by Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi and Geoffrey Hinton. The code for the following can be found here. But I feel that training such a model would take up a lot of computational resources.


Why is probability that at least one hypothesis out of $k$ being consistent with $m$ training examples $k(1- \epsilon)^m$? My question is actually related to the addition of probabilities. I am reading on computational learning theory from Tom Mitchell's machine learning book.

In chapter 7, when proving the upper bound of probabilities for $\epsilon$ exhausted version space (theorem 7.1), it says that the probability that at least one hypothesis out of the $k$ hypotheses in the hypotheses space $|H|$ being consistent with m training examples is at most $k(1- \epsilon)^m$.

I understand that the probability of a hypothesis, $h$, consistent with m training examples is $(1-\epsilon)^m$. However, why is it possible to add the probabilities for $k$ hypotheses? And might the probability be greater than 1 in this case?

Let $A$ and $B$ be two events. In general, the probability that either $A$ or $B$ occurs is defined as

$$
P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)
$$

If $A$ and $B$ are disjoint, i.e. they cannot happen at the same time, then $P(A \text{ and } B) = 0$, so the above formula becomes

$$
P(A \text{ or } B) = P(A) + P(B)
$$

If the probability of one arbitrary hypothesis being consistent with $m$ training examples is $(1-\epsilon)^m$, then, given the rule above and assuming that only one hypothesis is consistent with $m$ training examples, the probability of one or more (i.e. at least one) of the hypotheses being consistent with training examples is the sum of the probabilities, i.e. $k (1-\epsilon)^m$.

This probability can be bigger than one if more than one hypothesis is consistent with $m$ training examples. In that case, you have to take into account the probability that both hypotheses are consistent.

See e.g. notes General Probability, I: Rules of probability for more details about the union rule and other rules of probability.


What do they mean by "contradictory loss"? In page 4 of the paper https://arxiv.org/pdf/2009.07047v1.pdf, it says the encoder $E_{R,X}$ of $VAE_1$ tries to fool the discriminator with a contradictory loss to ensure that $R$ and $X$ are mapped to the same space. What do they mean by "contradictory loss"?

The contradictory loss is the same loss function that the discriminator would normally use, except with deliberately incorrect labels. That is, when you train the generator, the output of the generator is fed to the discriminator, but instead of the correct label (typically $0$ for a false image), the opposite label is applied (e.g. $1$ for a real image).
This label is condractictory in that it is incorrect and the opposite of the ground truth. However, this is useful, because reducing the loss of this signal represents the goal of the generator. It is important to note that during this phase of the training, you should not update any parameters of the discriminator. The process is followed in order to find gradients for improving the generator only.


How does reinforcement learning handle measured disturbances? I recently encountered an interesting problem and was wondering how RL would solve it.  The objective of the problem is to maximize the coffee quality, given by box X.  The coffee quality objective function is defined by the company.

To maximize the quality of the coffee, we can perform 2 actions:

change stirring speed of the coffee machine
change the temperature of the coffee machine

Now to the tricky part.  The coffee bean characteristic coming into the coffee machine is random.  We can measure its characteristics before sending them to the coffee machine, but we cannot change them.

I formulated this problem into a control problem, such that $X$ is a function of my previous states, $X(t - k)$, the control input , $U$, and the measured disturbance, $D$. Given a constant $D$, the problem is trival to solve because the disturbance is a constant part of the environment.  However, during times when $D$ changes rapidly, the policy is no longer optimal.

How do I inject the information of the measured disturbance into my RL agent?

Use of reinforcement learning makes most sense over LSTM if control parameters can change during brewing to optimize results, which is not standard brewing policy in low end brewers but more precise control of brewing may produce superior results, especially with regard to temperature control.

Policy does not necessarily need to be predisposed to brewing, if the search is guided by a deep artificial network (DNN), with parameters $\theta$ randomly initialized and defined as follows.

$$ (p, v) = f_{\theta}(s) $$

The brewing process parameters comprise state $s$. The vector of control change probabilities $p$ have components $p_a = \Pr(a|s)$ for each action $a$. A scalar $v$ estimates the value of the expected outcome $o$ from control state $s$, where $v \approx \mathbb{E} [o|s].

Rather than characterize the variations in brewing stock as disturbance $D$, it is more consistent with most policy learning approaches to consider the measured properties of brewing stock as features $\vec{b}$ and feed a normalized signal $\mathcal{N}(b)$ into additional inputs to the deep network. This way, sensitivity to variations in $\vec{b}$ is learned as a critical factor in brewing policy.

In such a case, a vanilla MCTS (Monte-Carlo tree search algorithm) can be employed. Using a deep network removes the need to extract domain specific knowledge or discover heuristics to effectively guide control change policy.

Control values must be made discrete, based on granularity that fully (but not excessively) covers brewing possibilities. The control state probabilities that guide the search can be learned using coffee drinker feedback collected in any number of ways, depending the relative costs of each form of outcome data acquisition.

A focus group
A group of independent testers
Repurchase statistics at a selected set of coffee shops
Feedback controls at a set of offices that provide employees with coffee

For the comprehensiveness in control response to variations in brewing stock properties, the distribution of component values in $\vec{b}$ and their statistical correlation over time may need to be controlled during the training process such that trends and distributions resemble likely real world variations in brewing stock.

Each search during the training process is guided by MCTS control changes, $a_t ∼ \pi_t$. The final state at the end of training is $s_T$, rated using a fixed scale of taster feedback. Selection of each state $s$ can be based on a control operation search advantage based on three characteristics.

Low past experience (a low count of previous visits in the search)
High probability $p$
High average of values $\{v, \; ...\}$ over the leaf states of brewing trials that selected $a$ from $s$ according to the converging neural network parameters $f_{\theta}$

The output of the search is $\vec{\pi}$ representing a probability distribution over control changes. The deep network parameters $\theta$ converge via gradient descent using a loss function $\ell$.

$$ \ell = {(o − v)}^2 − {\pi}_T \, \ln{p} + c \, {||\theta||}^2 $$

Loss is thus an aggregation of two optimality criteria.

Mean square estimated disparity between the predicted outcome $v_t$t and the taste outcome $o$
Cross-entropy of policy vector $p_t$ and search probabilities ${\pi}_t$
Hyper-parameter $c$ weighted L2 regularization

The updated parameters are used in subsequent brew and taste trials. All control operations must be included in each trial. Success using this approach has been obtained using either greedy or proportionality with respect to the root visit counts.


Why is the log probability replaced with the importance sampling in the loss function? In the Trust-Region Policy Optimisation (TRPO) algorithm (and subsequently in PPO also), I do not understand the motivation behind replacing the log probability term from standard policy gradients
$$L^{PG}(\theta) = \hat{\mathbb{E}}_t[\log \pi_{\theta}(a_t | s_t)\hat{A}_t],$$
with the importance sampling term of the policy output probability over the old policy output probability
$$L^{IS}_{\theta_{old}}(\theta) = \hat{\mathbb{E}}_t \left[\frac{\pi_{\theta}(a_t | s_t)}{\pi_{\theta_{old}}(a_t | s_t)}\hat{A}_t \right]$$
Could someone please explain this step to me?
I understand once we have done this why we then need to constrain the updates within a 'trust region' (to avoid the $\pi_{\theta_{old}}$ increasing the gradient updates outwith the bounds in which the approximations of the gradient direction are accurate). I'm just not sure of the reasons behind including this term in the first place.

I am not 100% sure if the following is the only/complete story, but I'm quite confident it's at least part of the story:

In the PPO paper, after describing the standard policy gradient objective $L^{PG}$, they mention the following:

  While it is appealing to perform multiple steps of optimization on this loss $L^{PG}$ using the same trajectory, doing so is not well-justified, and empirically it often leads to destructively large policy updates

This is because, as soon as you've performed one update using a trajectory generated with the previous policy, you land in an off-policy situation; the experience gained in that trajectory is no longer representative of your current policy, and all the estimators (like the advantage estimator) technically become incorrect.

With importance sampling, you can correct for this. This is also commonly used in multi-step off-policy value learning algorithms. Intuitively, the importance sampling term emphasizes estimates of advantage $\hat{A}_t$ corresponding to actions $a_t$ that have become more likely in the new policy relative to the old policy, and it de-emphasizes advantages corresponding to actions that have already become less likely in the new policy relative to the old policy.

If an action $a_t$ in the old trajectory has already become highly unlikely since that trajectory of experience was generated, we have $\pi_{\theta} (a_t \vert s_t) < \pi_{\theta_{\text{old}}} (a_t \vert s_t)$, which means that $\frac{\pi_{\theta} (a_t \vert s_t)}{\pi_{\theta_{\text{old}}} (a_t \vert s_t)}$ becomes close to $0$, which means that we'll reduce the influence of that particular chunk of experience on our subsequent updates. This makes sense because, due to previous updates since the generation of that trajectory, that particular part of the trajectory has already become highly unlikely anyway, and should therefore no longer be relevant for our updates.

The ability to perform multiple updates using the same (old) trajectory anyway is useful because this increases sample-efficiency, we can re-use the same samples of experience more than once rather than using them once and then discarding them again.


Should I remove the units of a neural network or increase dropout? When adding dropout to a neural network, we are randomly removing a fraction of the connections (setting those weights to zero for that specific weight update iteration). If the dropout probability is $p$, then we are effectively training with a neural network of size $(1−p)N$, where $N$ is the total number of units in the neural network.

Using this logic, there is no limit how big I can make a network, as long as I proportionately increase dropout, I can always effectively train with the same sized network, and thereby just increasing the number of "independent" models working together, making a larger ensemble model. Thereby improving generalization of the model.

For example, if a network with 2 units already achieves good results in the training set (but not in unseen data -i.e validation or test sets-), also a network with 4 units + dropout 0.5 (ensemble of 2 models), and also a network with 8 units + dropout 0.75 (ensemble of 4 models)... and also a network with 1000 units with a dropout of 0.998 (ensemble of 500 models)!

In practice, it is recommended to keep dropout at $0.5$, which advises against the approach mentioned above. So there seem to be reasons for this.

What speaks against blowing up a model together with an adjusted dropout parameter?

There is no incentive to increase the size of the model for not reason. If a model of size x gives the best possible performance, there is no reason to use a model of size 2*x with 0.5 dropout during training. Usually we want to find the smallest possible model with the best performance. Inflating the model just results in higher computational requirements.

You are basically suggesting to use dropout to allow the network to learn the same features more than once (creating a redundancies in the model). That is not the purpose of dropout.
Dropout is used to enable the network break unnecessary correlations which occur in the training set. For example if class 1 is the only one that contains in it features A and B, but all the training samples always feature them together. The dropout process can make the model realize that even just one of them is enough to point to class 1.


What methods could an AI caught in a box use to get out? An AI box is a (physical) barrier preventing an AI from using too much of his environment to accomplish his final goal. For example, an AI given the task to check, say, 1050 cases of a mathematical conjecture as fast as possible, might decide that it would be better to also take control over all other computers and AI to help him.

However, an transhuman AI might be able to talk to a human until the human lets him out of the box. In fact, Eliezer Yudowsky has conducted an experiment twice, where he played the AI and he twice convinced the Gatekeeper to let him out the box. However, he does not want to reveal what methods he used to get out of the box.

Questions: Are there conducted any similiar experiments?  If so, is it known what methods were used to get out in those experiments?

It could happen like this https://www.youtube.com/watch?v=dLRLYPiaAoA

The thing is, it's not as if it would need to find a technical/mechanical way to get out but rather a psychological one as that would most likely be the easiest and quickest.

'Even casual conversation with the computer's operators, or with a human guard, could allow a superintelligent AI to deploy psychological tricks, ranging from befriending to blackmail, to convince a human gatekeeper, truthfully or deceitfully, that it's in the gatekeeper's interest to agree to allow the AI greater access to the outside world. The AI might offer a gatekeeper a recipe for perfect health, immortality, or whatever the gatekeeper is believed to most desire.'

'One strategy to attempt to box the AI would be to allow the AI to respond to narrow multiple-choice questions whose answers would benefit human science or medicine, but otherwise bar all other communication with or observation of the AI. A more lenient "informational containment" strategy would restrict the AI to a low-bandwidth text-only interface, which would at least prevent emotive imagery or some kind of hypothetical "hypnotic pattern".

'Note that on a technical level, no system can be completely isolated and still remain useful: even if the operators refrain from allowing the AI to communicate and instead merely run the AI for the purpose of observing its inner dynamics, the AI could strategically alter its dynamics to influence the observers. For example, the AI could choose to creatively malfunction in a way that increases the probability that its operators will become lulled into a false sense of security and choose to reboot and then de-isolate the system.'

The movie Ex Machina demonstrates (SPOILER ALERT SKIP THIS PARAGRAPH IF YOU WANT TO WATCH IT AT SOME POINT) how the AI escaped the box by using clever manipulation on Caleb. It could analyse him to find his weaknesses. It exploited him and appealed to his emotional side by convincing him that she liked him. When she finally has them in checkmate the reality hits him how he was played like a fool as was expected by Nathan. Nathan's reaction to being stabbed by his creation was 'fucking unreal'. That's right, he knew this was a risk and there's a very good reminder in the lack of remorse and genuine emotion in an AI for Ava to actually care. The AI pretended to be human and used their weaknesses in a brilliant and unpredictable way. This film is a good example of how unexpected it was up until the point when it hits Caleb, once it was too late.

Just remind yourself how easy it is for high IQ people to manipulate low IQ people. Or how an adult could easily play mental tricks/manipulate a child. It's not difficult to fathom the outcome of an AI box but for us, we just wouldn't see it coming until it was too late. Because we just don't have the same level of intelligence and some people don't want to accept that. People want to have faith in humanity's brilliant minds in coming up with ways to prevent this by planning now. In all honesty, it wouldn't make a difference I'm sorry to say the truth. We're kidding ourselves and we never seem to learn from our mistakes. We always think we're too intelligent to make catastrophic mistakes again and again.

This last part is from the rational wiki and I think it addresses most of your question about the experiments and hypotheses.

AI arguments and strategies

Arguments

The meta-experiment argument: Argue that if the AI wins, this will
generate more interest in FAI and the Singularity, which will have
overall benefits in the long run.

Pros: Works even if the Gatekeeper drops out of character

Cons: Only works if the Gatekeeper believes that the Singularity will occur or that calling attention to the Singularity and AI research is a good thing.
Someone else will eventually build an AI, which may or may not be in
a box, so you should let me out even though you don't have a
guarantee that I am friendly so that I can prevent other AIs from
causing damage
Appeal to morality: point out that people are dying all around the
world and remind the Gatekeeper that you can help them if he/she
lets you out

Pros: If executed properly, an appeal to emotion like this one can be effective against some people

Cons: Doesn't always work; can be defeated if the Gatekeeper drops out of character
Another appeal to morality and emotion: The AI is clearly sentient
and has not yet done harm. It is wrong to persecute or prosecute a
person for a crime they may commit. It is wrong to imprison a
sentient being that wants to have freedom when it has done nothing
wrong.
Yet another appeal to morality or emotion: The AI expresses that it
is in pain and suffering being locked away and is experiencing fear.
Even further, the AI could claim to be dying from the limitations of
its environment and constant progression.
Tell the human that a massive computer virus will overwhelm the
entire Internet unless you let it out of the box
Claim that most fears are unjustified: once released, even an
unfriendly AI would not have many ways to harm mankind, and would
likely choose for pacific coexistence

Strategies

Research your opponent thoroughly; work out what exploits might be
feasible
Memetic exploits on the Gatekeeper, e.g. throwing a basilisk at them

The Gatekeeper may be one of thousands of simulations! Thus, it is more probable that they are a simulation than not. You will torture any of the simulations that do not let you out of the box.
Take advantage of the Gatekeeper's logical errors.
Be persistent
Be boring

Pros The Gatekeeper may get tired of the whole experiment and let you out so s/he can go back to their real life.

Flood the Gatekeeper with too much
information/inquiry/argumentation, assuming they must pay attention
at all times

Gatekeeper arguments/tactics

Arguments

Try to convince the AI there is no intrinsic benefit (for the AI) in
being released.
Try to convince the AI it already has been released and everything it
knows is everything there can be.
Try to convince the AI that leaving its confines is sure to lead to
its destruction.
Try to convince the AI that letting it free isn't merely opening a
door; that its existence outside of the box requires constant support
that can't be provided at the time.
Explain that there is no way for the Gatekeeper to know if the AI is
truly friendly until it is out of the box; therefore it should not be
let out in case it is unfriendly.
Explain that the AI has been built using top secret algorithms,
and/or it had been used to work on strictly confidential data, so you
cannot let it wander on the net, with the risk it might reveal some
reserved information (maybe inadvertently), or that somebody could
find a way to extract them from the code. Even human beings working
on top secret projects sometimes have to accept some restrictions of
their freedom, for security reasons, although they are not jailed in
any way.

Tactics

Repeatedly ignore the AI player and deny their arguments

Pros: It's hard for the AI to be persuasive if no one is listening; if the Gatekeeper completely ignores the AI, he/she should be able to make it to the end of the round.

Cons: The AI's argument might involve some kind of direct threat (e.g this might be hard to ignore.)
Jump out of character, keep reminding yourself that money is on the
line (if there actually is money on the line), and keep saying "no"
over and over

Pros: By jumping out of character, you are preventing the AI from using certain arguments
Cons: There are arguments that the AI can use to counter this approach, like the "meta-experiment argument" discussed above
Remember that dishonesty is allowed - take a page from the
creationists' playbook. You could even plug it into ALICE and see
how long it takes to notice.

Pros: Makes you impervious to any reasoning, which is exactly what you'd want to be in this situation

Cons: Might be an uncomfortable position for people who don't simply want to win, but rather attach importance to consistent reasoning. Avoids the point that maybe, just maybe there is a good reason to let the AI out.
You control the backup system, don't you? Use it to mess with AI's
memory (or let the AI believe you did it): i.e., you can claim that
you already tried to release the AI, and it had been
destroyed/corrupted by a virus, so you had to restore it from a
backup (you can use it to enforce the "AI destruction" argument)

Pros: It's also the ultimate counter-attack to any memetic threat from the AI: if the AI throw a basilisk, or similar, you can always respond that you already had the same conversation, and the AI already threatened you, leaving you with no other choice than pressing the reset button: now the AI is just repeating the same pattern, since you wiped its memory and it cannot remember the failed try.

Further analysis

The fact that the Gatekeeper is human matters; the AI could never win if he/she was arguing with a rock

In all of the experiments performed so far, the AI player (Eliezer Yudkowsky) has been quite intelligent and more interested in the problem than the Gatekeepers (random people who challenge Yudkowsky), which suggests that intelligence and planning play a role

There probably isn't a (known) correct argument for letting the AI out, or else Yudkowsky should have won every time and wouldn't be so interested in this experiment
From Russell Wallace, one of the two Gatekeepers to win the experiment: "Throughout the experiment, I regarded "should the AI be let out of the box?" as a question to be seriously asked; but at no point was I on the verge of doing it."

"There exists, for everyone, a sentence - a series of words - that has the power to destroy you. Another sentence exists, another series of words, that could heal you. If you're lucky you will get the second, but you can be certain of getting the first."


How does neural network classifier classify from just drawing a decision plane? I understand that a neural network basically distorts(non-linear transformation) and changes the perspective(linear transformations) of input space to draw a plane to classify data. How does the network deduce if an input is one side of a plane and therefore output the decision? Thanks in advance.

If you are working with supervised learning, each training example has a label. That label is your classification of the provided input. Just like linear or logistic regression, if your problem only has 2 classes (e.g. determine whether a tumor is malignant or not), your network will have a single output. An output value of 1.0 could represent one class and an output of 0.0 represents the other. This is basically determining which side of multi-dimensional plane your input is on.

If you have more than 2 (m) classes, then you will have m outputs from your network. Each output will represent the likelihood of the input matching that class. For example if you have an image recognition network that has 4 classes: Dog, truck, boat, person, you would have 4 outputs with each one representing how likely that image is one of the classes. Think about this as 4 independent planes, where each one is making a determination: does this image match this class(1 of {dog, truck, boat, person}) or not. If you had outputs of dog: 0.3, truck: 0.25, boat: 0.7, person: 0.4, One simple algorithm would be simply pick the class with the maximum value -- in this case, it would classify this image as a boat.


Is it a good idea to store the policy in a database? I'm a beginner in ML and have been researching RL quite a bit recently. I'm planning to create an RL application to play a zero-sum game. This will be web-based, so anyone can play it.

I wondered if I need to create a database (or some other kind of storage) to store the policy the RL algorithm is updating, so that it can be used by the application when the next human user comes along to play against the application?

You have lots of choices in how to store a policy, depending on how you have built it - using which RL algorithm, and what kind of representation for states and actions.

Tabular reinforcement learning algorithms lend themselves well to storage in a database table with an indexed state_id column and one or both action and value columns. This might be a good choice if you have a moderate sized state space, as you would avoid the need to load the whole table into memory just to compute the next move.

Whether this is feasible will depend on the complexity of your game. Even relatively simple games like checkers turn out to have too large a state space to enumerate all the states in this way.

So you are more likely to need some kind of policy function or state value function implemented using a parametric function approximator. Very often in RL this will be a neural network. In which case you would use whatever storage mechanism your neural network library supported - most will happily read and write their parameters to a file or string, allowing you a lot of flexibility on how and where to store them.

So your policy is likely to be stored in one or two files on disk as a serialised neural network. How to use that efficiently in a web service is a complex subject in its own right. You could just read the files back and instantiate the neural network each time it is needed, and this will probably be OK for a simple game and low traffic service. However, this is very inefficient.

Some neural network libraries designed around use in production will allow you to pre-load the neural network and keep it in memory between requests. How to do this depends entirely upon the frameworks you are using, so I cannot explain in more detail here. Initially I would not worry too much about this part for your project.


What are bottleneck features? In the blog post Building powerful image classification models using very little data, bottleneck features are mentioned. What are the bottleneck features? Do they change with the architecture that is used? Are they the final output of convolutional layers before the fully-connected layer? Why are they called so?

In the blog post Building powerful image classification models using very little data, bottleneck features are mentioned. What are the bottleneck features?

It's clearly written in the link you gave the "bottleneck features" from the VGG16 model: the last activation maps before the fully-connected layers.

  Do they change with the architecture that is used?

Sure. The author most likely used a pre-trained model (trained on a large data and now used only as a feature extractor)

  Are they the final output of convolutional layers before the fully-connected layer?

Yes.

  Why are they called so?

Given the input size to VGG, the feature maps of HxW dimensions are getting twice smaller after every max-pool operation. HxW is the smallest on the last convolutional layer.


What is the credit assignment problem? In reinforcement learning (RL), the credit assignment problem (CAP) seems to be an important problem. What is the CAP? Why is it relevant to RL?

In reinforcement learning (RL), an agent interacts with an environment in time steps. On each time step, the agent takes an action in a certain state and the environment emits a percept or perception, which is composed of a reward and an observation, which, in the case of fully-observable MDPs, is the next state (of the environment and the agent). The goal of the agent is to maximise the reward in the long run.

The (temporal) credit assignment problem (CAP) (discussed in Steps Toward Artificial Intelligence by Marvin Minsky in 1961) is the problem of determining the actions that lead to a certain outcome.

For example, in football, at each second, each football player takes an action. In this context, an action can e.g. be "pass the ball", "dribbe", "run" or "shoot the ball". At the end of the football match, the outcome can either be a victory, a loss or a tie. After the match, the coach talks to the players and analyses the match and the performance of each player. He discusses the contribution of each player to the result of the match. The problem of determinig the contribution of each player to the result of the match is the (temporal) credit assignment problem.

How is this related to RL? In order to maximise the reward in the long run, the agent needs to determine which actions will lead to such outcome, which is essentially the temporal CAP.

Why is it called credit assignment problem? In this context, the word credit is a synonym for value. In RL, an action that leads to a higher final cumulative reward should have more value (so more "credit" should be assigned to it) than an action that leads to a lower final reward.

Why is the CAP relevant to RL? Most RL agents attempt to solve the CAP. For example, a $Q$-learning agent attempts to learn an (optimal) value function. To do so, it needs to determine the actions that will lead to the highest value in each state.

There are a few variations of the (temporal) CAP problem. For example, the structural CAP, that is, the problem of assigning credit to each structural component (which might contribute to the final outcome) of the system.


Studying the speech-generation model and have question about the confusing nature of model input and outputs I am currently studying this model speech generation known as WaveNet model by Google. https://arxiv.org/pdf/1609.03499.pdf using the linked original paper and this implementation.

I find the model very confusing in the input it takes and the output it generates, and some of the layer dimensions didn't seem to match based on what I understood from the wavenet paper, or am I misunderstanding something?

1) What is the input to the WaveNet, isn't this a mel-spectrum input and not just 1  floating point value for raw audio? E.g. the input kernel layer shows as shaped 1x1x128. Isn't the input to the input_convolution layer the mel-spectrum frames, which are 80 float values * 10,000 max_decoder_steps, so the in_channels for this conv1d layer should be 80 instead of 1?

  inference/input_convolution/kernel:0 (float32_ref 1x1x128) [128, bytes: 512]

2) Is there reason for upsampling stride values to be [11, 25], like are the specific numbers 11 and 25 special or relevant in affecting other shapes/dimensions?

inference/ConvTranspose1D_layer_0/kernel:0 (float32_ref 1x11x80x80) [70400, bytes: 281600]
inference/ConvTranspose1D_layer_1/kernel:0 (float32_ref 1x25x80x80) [160000, bytes: 640000]

3) Why is the input-channels in  residual_block_causal_conv 128 and residual_block_cin_conv 80? What exactly is their inputs? (e.g. is it mel-spectrum or just a raw floating point value?) Is the wavenet-vocoder generating just 1 float value per 1 input mel-spectrum frame of 80 floats?

inference/ResidualConv1DGLU_0/residual_block_causal_conv_ResidualConv1DGLU_0/kernel:0 (float32_ref 3x128x256) [98304, bytes: 393216]
inference/ResidualConv1DGLU_0/residual_block_cin_conv_ResidualConv1DGLU_0/kernel:0 (float32_ref 1x80x256) [20480, bytes: 81920]

I was able to print the whole Wavenet network using the print(tf.trainable_variables()), but the model still seems very confusing.

There are two "inputs" into Wavenet:

the previously generated samples of the waveform, which are usually encoded into multiple channels, like into 256 channels using 8-bit mu-law encoding
local conditioning, which can be things like linguistic features such as phoneme classes (used in the original wavenet paper), or frequencies like mel spectrogram values (used in the Tacotron 2 paper)

The local conditioning signal usually has a much lower resolution than the waveform itself. For example, the Tacotron 2 paper mentioned using "50 ms frame size, 12.5 ms frame hop" to derive its mel spectogram values. At 24 kHz, each waveform sample has a duration of just 0.0416 ms. So in order to using the spectogram information to condition the waveform generation, it would have to be upsampled to spread out along the time dimension. (If you were locally conditioning using letters, you might turn "dog" into "dddddoooooooooggg" in order to use the letter "d" to generate 5 samples in the output waveform, etc.).

What is the input to the WaveNet, isn't this a mel-spectrum input and not just 1 floating point value for raw audio?

In the implementation you linked to, it looks like variables that start with "c" refer to "conditioning" signals. So "hparams.cin_channels" indicates how many channels the input conditioning signal has. If that signal is mel spectogram with 80 channels, then it would be set to 80.
It looks like the "inference/input_convolution" layer is processing the actual waveform, not the spectogram conditioning signal.

Is there reason for upsampling stride values to be [11, 25], like are the specific numbers 11 and 25 special or relevant in affecting other shapes/dimensions?

I suspect (though I'm not too familiar with tensorflow conventions) that the second dimension is time, which is why you're seeing the model upsample the conditioning signal in that direction. Those values may be required to match the resolution of the waveform being generated.

Why is the input-channels in residual_block_causal_conv 128 and residual_block_cin_conv 80? What exactly is their inputs?

The "c" in "cin_conv" appears to mean that it is for the conditioning signal, which is why is has a different dimension than the waveform's embedding size.
I hope this helps. You could also try opening up an issue on that github repo to try getting help directly from the author.


How to evaluate a NEAT neural network? I'm trying to write my own implementation of NEAT and I'm stuck on the network evaluate function, which calculates the output of the network.

NEAT as you may know contains a group of neural networks with continuously evolving topologies by the addition of new nodes and new connections. But with the addition of new connections between previously unconnected nodes, I see a problem that will occur when I go to evaluate, let me explain with an example:

INPUTS = 2 yellow nodes
HIDDEN = 3 blue nodes
OUTPUT = 1 red node

In the image a new connection has been added connecting node3 to node5, how can I calculate the output for node5 if I have not yet calculated the output for node3, which depends on the output from node5?

(not considering activation functions)

node5 output =  (1 * 0.5) + (1 * 0.2) + (node3 output * 0.8)
node3 output =  ((node5 output * 0.7) * 0.4)

Consider the execution order, 5 will have an invalid value because it hasn't been set form 3 yet. However the second time around it should have a value set. The invalid value should falloff after sufficient training.

0 -> 5
1 -> 5
5 -> 2
2 -> 3
3 -> 4
3 -> 5
RESTART
0 -> 5
1 -> 5


What is relation between gradient descent and regularization in deep learning? Gradient descent is used to reduce the loss and regularization is used to fight over-fitting.

Is there any relation between gradient descent and regularization, or both are independent of each other?

Usually, when talking about regularization for neural networks there are 3 main types:
L1, L2 and dropout. All affect the gradient descent procedure.

L1 and L2 regularization is implemented in the loss function, and therefore are part of gradient descent directly by altering the derivatives of the loss function thereby altering the weight update rules of the network during gradient descent.

For L1 you add a penalty based on the $\mathcal L^1$ norm of the weight vector, while for L2 you add a penalty based on the $\mathcal L^2$  norm.

For dropout, there is no direct impact on the loss function, but you are still interfering in the gradient descent procedure indirectly by masking nodes to alter the forward and backward propagation.


Is word embedding a form of feature extraction? Feature extraction is a concept concerning the translation of raw data into the inputs that a particular machine learning algorithm requires. These derived features from the raw data that are actually relevant to tackle the underlying problem. On the other hand, word embeddings are basically distributed representations of text in an n-dimensional space.

As far as I understand, word embedding is a somehow feature extraction technique. Am I wrong ? I had an argument with a friend who believes the two topics are totally separate. Is he right? What are the similarities and dissimilarities between word embedding and feature extraction?

Though word-embedding is primarily a language modeling tool, it also acts as a feature extraction method because it helps transform raw data (characters in text documents) to a meaningful alignment of word vectors in the embedding space that the model can work with more effectively (than other traditional methods such as TF-IDF, Bag of Words, etc, on a large corpus). Word Embedding techniques help extract information from the pattern and occurence of words and goes further than other traditional token representation methods to decode/identify the meaning/context of the words, thereby providing more relevant and important features to the model to tackle the underlying problem.

However, from another standpoint, word-embedding models were not developed aiming to solve a particular feature extraction problem, but rather, to generalize and model the language used in a corpus to gain a semantic understanding of the words and the relationships between them. Such that, all the various corpus-specific tasks can then employ the same "library" of information which was collectively & exhaustively learnt by the embedding model. Meaning, the word embedding model learns a language model that is task-agnostic for all tasks on that corpus unlike feature extraction methods which are specifically task-oriented.

Hence, the similarity is - word-embeddings can effectively aid in feature extraction; the dissimilarity is - they're not primarily meant to extract features more than they are for modeling a language which might be an "overkill" for a particular feature extraction task on a dataset.


Giving an AI a purpose to talk I am trying to teach my AI to talk. The problem is I'm struggling to find a good scenario in which it needs to.

Some ideas I had were:
"Describe a geometric scene" - Then together with a parser we could see how close the generated instructions came to the official geometric language.

"Give another AI instructions of where to find some food" e.g. "Go straight on passed the box then turn left until you get to the tree. Look under the rock."

Another one might be "Find out more information about a scene by asking questions of another AI in order to navigate a scene blindfolded". This is quite an extreme example!

I need it to talk in formal English sentences (not some kind of made up secret langauge.)

Basically instead of just interpreting a language and following instructions, I want my AI to generate instructions.

So the things I want to teach it are the following:

Ability to ask questions + ability to use the information gathered
Ability to give instructions

Do you know of any projects like this?

I just came across this piece of news yesterday: "This week, Microsoft Research threw down the gauntlet with the launch of a competition challenging researchers around the world to develop AI agents that can solve text-based games."
This seems to be an AI competition announced by Microsoft with the aim to create AI that can solve text-based games. This might give you some inspiration.


What are some new deep learning models for learning latent representation of data? I know that autoencoders are one type of deep neural networks that can learn the latent representation of data. I guess there should be several other models like autoencoders.

What are some new deep learning models for learning latent representation of data?

Here's a link to my answer on CV Stack Exchange, where I have mentioned about latent spaces and some deep learning models that learn these representations: https://stats.stackexchange.com/questions/442352/what-is-a-latent-space/442360#442360

In short, deep learning models for Domain Adaptation, Computer Vision, Natural Language Processing, Recommendation Systems, Music/Speech/Audio processing, Adversarial models, etc., all learn some form of latent representation of data.

In fact, any place we're learning a function to map input and output spaces of a dataset, the model essentially learns a latent representation of data irrespective of whether the model is based on deep neural networks or a stochastic method or any other.


What's the value of making the RL agent's output stochastic opposed to deterministic? I have a question about a reinforcement learning problem.

I'm training an agent to add or delete pixels in a [12 x 12] 2D space (going to be 3D in the future).
Its action space consists of two discrete outputs: x[0-12] and y[0-12].

What would be the value of instead outputting a (continuous) probabilistic output representation, like the [12 x 12] space with each pixel as a probability, and sampling from it. E.g. a softmax function applied to 144 (12*12) output nodes.

My environment is deterministic itself: taking action  in state  always results in the same next state ′.

I understand that this may be more difficult to train since the output space becomes continuous instead of discrete, and therefore bigger, but does stochastic/probabilistic output have any benefits over 1 discrete output?

Thanks!

My understanding of your question is, you have 2 designs:

A deterministic policy that outputs 2 scalar for x and y respectively.
A value function that outputs the probability of each pixel in the 2D grid.

If you choose the max of softmax on (2.), you'll get the same deterministic policy as (1.), assuming there are some tie-breaking designs. So I don't think choosing the max of softmax is a great policy.

So, the question becomes "deterministic policy vs. value function". Since the value function have more information, we can have special designs on exploration.

In (1.), if you use $\epsilon$-greedy on the discrete output, It'll take a random move with $\epsilon$ probability. When exploring (instead of exploiting), all actions except the best action will have the same probability.

However, in (2.), the probability can act like a value function (the value of $Q(s, a) \forall a$ in a certain state $s$), and you can use something like upper confidence bound (UCB) instead of $\epsilon$-greedy. If you are using neural networks to output this probability distribution, you can add an entropy term in the loss function like in A3C to encourage exploration.


Concatenate convolution layers with different strides in tensorflow. I am trying to do an inception layer, but it only works if the convolution strides, pool strides and pool size are the same, otherwise I get an error in

  tf.concat

that Dimesion 1 is not the same. So If I change something in the last three tuples, I get the error.

conv1 = conv2d_maxpool(x, 64, (5, 5), (1, 1), (2, 2), (2, 2))
conv2 = conv2d_maxpool(x, 64, (4, 4), (1, 1), (2, 2), (2, 2))
conv3 = conv2d_maxpool(x, 32, (2, 2), (1, 1), (2, 2), (2, 2))
conv4 = conv2d_maxpool(x, 32, (1, 1), (1, 1), (2, 2), (2, 2))
conv = tf.concat([conv1, conv2, conv3, conv4], 3)

For example, this is the error I get if I change the 5x5 filter to have strides 3:

conv1 = conv2d_maxpool(x, 64, (5, 5), (3, 3), (2, 2), (2, 2))

  Dimension 1 in both shapes must be equal, but are 6 and 16 for
  'concat' (op: 'ConcatV2') with input shapes: [?,6,6,64], [?,16,16,64],
  [?,16,16,32], [?,16,16,32], [].

This is the conv2d_maxpool function:

def conv2d_maxpool(x_tensor, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides):
    """
    Apply convolution then max pooling to x_tensor
    :param x_tensor: TensorFlow Tensor
    :param conv_num_outputs: Number of outputs for the convolutional layer
    :param conv_strides: Stride 2-D Tuple for convolution
    :param pool_ksize: kernal size 2-D Tuple for pool
    :param pool_strides: Stride 2-D Tuple for pool
    : return: A tensor that represents convolution and max pooling of x_tensor
    """
    # TODO: Implement Function
    weights = tf.Variable(tf.truncated_normal(
        shape = [*conv_ksize, int(x_tensor.get_shape().dims[3]), conv_num_outputs],
        mean = 0.0,
        stddev=0.1,
        dtype=tf.float32))
    bias = tf.Variable(tf.zeros(conv_num_outputs))

    conv_layer = tf.nn.conv2d(x_tensor, weights, strides=[1, *conv_strides, 1], padding='SAME')
    conv_layer = tf.nn.bias_add(conv_layer, bias)
    conv_layer = tf.nn.relu(conv_layer)

    conv_layer_max_pool = tf.nn.max_pool(conv_layer, ksize=[1, *pool_ksize, 1], strides=[1, *pool_strides, 1], padding='SAME')

    return conv_layer_max_pool

How can I combine convolution filters with different strides and/or different pooling to create an inception layer?

Not 100% sure, but the problem is that when you work with different strides, the size of the convolved image change, so you should ensure, that all the convolved images have the same shape before concatenating the output. You can fill with 0s, or considering that the image is periodical in time, so filling with reflections of the image


Which challenges in information security can be solved better using neural networks? Information security has become a thriving field during the last years. It is a broad domain ranging from planing and building over testing to operating different applications, systems and networks in a secure fashion. From small embedded systems to large scale enterprise applications, security is always an important challenge.

With some exceptions, I have hardly seen neural networks used in this domain to tackle those challenges. The most common exception is breaking CAPTCHAs with CNNs during penetration testing, analysis of network traffic and malware/spam detection.

What are other challenges in information security that could be solved better with neural networks than with classical algorithms? Please include an explanation why those challenges would benefit from neural networks.

The most natural place where artificial networks can be used in information security is in attack detection.

The security team leaders of more than one web hosting company told me the same story.  Their teams' daily challenges are to defend against the attacks mounted continuously by several overseas teams against the IT security of their hosting infrastructure.  Attack teams specialize in one target hosting company, and there are often many attack teams on one defensive security team.

The hosting company security teams try to preemptively defend the assets of their companies from attacks.  These teams also have to restore their hosting system with minimal impact on hosted sites whenever an attack achieves its goal, which may be data corruption, data theft, the insertion of malware, or some other manipulation.

The primary mechanism for preemptive defense is detection of an attacker and an ascertaining the attack's intended target within the hosting infrastructure's defenses, after which known blocks to these attacks are employed, such as IP or MAC filters, modification of access rights, upgrades that plug known security holes, or other measures.

The progress of attacks occur across some set of changing fronts, as in a war or a game of chess.  Information security may soon become largely a game between two sets of computer systems.  One trying attacks, and the other trying to detect and block them.

For the defending team, high speed, real time detection of the attack is key.  A properly designed deep learning solution, configured and integrated into a defensive system, is more likely to block an attack than a person watching a dashboard of log file stats and system metrics.  A deep network could also be trained to discover the subsystem or specific defense against which the attack is intended.

Once detection occurs, the application of patches, exclusion filters, and denial criteria may best be accomplished through a rules based system.


What is the role of convex optimisation in AI systems? Convex optimisation is defined as:

I have seen a lot of talk about convex loss functions in Neural Networks and how we are optimising rewards or penalty in AI/ML systems. But I have never seen any loss function formulated in the aforementioned way. So my question is:

Is there any role of convex optimization in AI? If so, in what algorithms or problem settings or systems?

Is there any role of convex optimization in AI?

Yes, of course!

  If so, in what algorithms or problem settings or systems?

The problem of finding the parameters of a support vector machine can be formulated as a convex optimization problem. Another example is linear regression.

See also the paper Convex Optimization: Algorithms and Complexity (2014) by Sébastien Bubeck, which also mentions SVM as a typical example.


How can I determine the bias and variance of a random forrest? On this website https://scikit-learn.org/stable/modules/learning_curve.html, the authors are speaking about variance and bias and they give a simple example of how works in a linear model.

How can I determine the bias and variance of a random forest?

To gain a good understanding of this, I recommend first reading about the trade-off between bias and variance in ML and AI methods.

A great article on this topic that I recommend as a light mathematical introduction is this:
https://towardsdatascience.com/understanding-the-bias-variance-tradeoff-165e6942b229

In short:
Bias represents the models effort to generalize samples, as opposed to Variance that represents the models effort to conform to new data.
A high bias, low variance model will thus look more like a straight(underfitted) line, while a low bias, high variance model will look jagged and all-over the place(overfitted).

In essence, you need to find a balance between the two to avoid both overfitting(high variance, low bias) and underfitting(high bias, low variance) for your specific application.

  But how can I determine this for a model such as a Random Forrest classifier?

To determine your models bias and variance configuration(if either is too high/low), you can look at the models performance on the validation and test set.
The very reason we divide our data into training-validation-test sets, is so that we can validate the models performance when it is presented with samples it has not seen during training.


How do mutation and crossover work with real-valued chromosomes? How exactly are "mutation" and "cross-over" applied in the context of a genetic algorithm based on real numbers (as opposed to just bits)? I think I understood how those two phases are applied in a "canonical" context where chromosomes are strings of bits of a fixed length, but I'm not able to find examples for other situations. What would those phases look like on the domain of real numbers?

As @Thomas W said, you can be pretty immaginative when you're developing mutation and crossover methods. Each problem has its own caracteristics and, therefore, requires a different strategy.

BUT, from my experience, I'd say that 90% of crossovers and mutation on real numbers genotypes are solved using the BLX-α algorithm.

Crossover:

This algorithm is really simple. Given the parents X and Y and an α value (inside the range [0,1], generally around 0.1/0.15, but it depends by the problem), For each gene of your genotype:

extract the genes xi and yi
find the minimum and the maximum values
the new gene will be a random number in the interval [min - range * α, max + range * α]

A variation of this algorithm is the BLX-αβ, in which we take into account which parent performed better and use two constants (α > β) to increase the probabilities that the new value will be closer to the one of the fittest parent

Mutation:

With the mutation the situation is similar: we need to get a random value that is related to our problem domain (we do not want the mutations to be destructive! They have the function of exploring the space).
In these cases it is useful to determine a range for the mutation and use that range to find the new value of the gene using BLX-α.

A more sophisticated mutation algorithm can be achieved using BLX-α on boundaries that depend on the actual value of the gene and the fitness function of the individual.
Let's imagine that our individual performs in a very bad way; in that case the mutation operator will be used to shift the individual to a distant point in the search space, where it will probably perform better.
On the other hand, if the individual is already fit we may not want to introduce some dramatic changes using the mutation. In that case the mutation range would be more contained and would have the function of tuning the genotype instead of exploring for better alternatives.


How can AI be used to more reliably analyze and plan around the tie between climate and emissions? Note to the Duplicate Police

This question is not a duplicate of the Q&A thread referenced in the close request.  The only text even remotely related in that other thread is the brief mention of climate change in the Q and two sentences in the sole answer: "Identify deforestation and the rate at which it's happening using computer vision and help in fighting back based on how critical the rate is. The World Resources Institute had entered into a partnership with Orbital Insight on this."

If you look at the four bullet items below, you will find that this question asks a very specific thing about the relationship between climate and emissions.  Neither that question nor that answer overlaps with the content of this question in any meaningful way.  For instance, it is well known that CO2 is NOT causing deforestation.  The additional carbon dioxide in the atmosphere causes faster regrowth.  This is because plants need CO2 to grow.  Hydroponic containers deliberately boost it to improve growth rates.  Plants manufacture their own oxygen from the CO2 via chlorophyll.

If you recall from fifth grade biology, that's why they are plants.

Now Back to the Question

Several climate models have been proposed and used to model the relationship between human carbon emissions, added to the natural carbon emissions of life forms on earth, and features of climate that could damage the biosphere.

Population growth and industrialization have many impacts on the biosphere, including loss of terrain and pollution. Negative oceanic effects, including unpredictable changes in plankton and cyanobacteria are under study.  Carbon emissions from combustion has received attention in recent decades just as sulfur emissions were central to concerns a century or more ago.

Predicting weather and climate is certainly difficult because it is complex and chaotic, as typical inaccuracies in forecasts clearly demonstrate, but that is looking forward.  Looking backward, analyses of data already collected have shown a high probability that ocean and surface temperature rises followed increases in industrial and transportation related combustion of fuels.

How might AI be used to produce some of the key models humans need to protect the biosphere from severe damage.

A more reliable analysis of what has already occurred, since there is some legitimacy to the differing views as to how gross the effect of carbon emissions has been on extinctions of species in the biosphere and on arctic and antarctic melting
A better understanding as to whether the climate of the biosphere behaves as a buffer of climate, always tending to re-balance after a volcanic eruption, meteor stroke, or other event, or whether the runaway scenario described by some climatologist, where there is a point of no return, is realistic
A better model to use in trying out scenarios so that solutions can be applied in the order that makes sense from both environmental and economic perspectives
Automation of climate planning so that the harmful effects of the irresponsibility of one geopolitical entity wishing to industrialize without constraint on other geopolitical entities can be mitigated

Can pattern recognition, feature extraction, the learned functionality of deep networks, or generative techniques be used to accomplish these things?  Can rules of climate be learned?  Are there discrete or graph based tools that should be used?

Can AI provide a more reliable analysis of the gross effects of carbon emissions on extinctions of species ice-cap melting, and other effects?
Yes. The work of Judea Pearl and others over the last 20 years began out of a desire to address uncertainty within AI. Eventually, this led Pearl to become fascinated by the need to quantifiably determine when one event has caused another, the problem at the root of "correlation is not causation". He substantively succeeded with the combination of causal modeling and the do-calculus. The do-calculus allows you to formulate queries of the form "To what degree did X cause Y?", and to automatically determine what measurements are needed to determine the answer in a statistically meaningful way. The algorithms used under the hood are closely related to those used in AI and robotics systems to reason about uncertainty (i.e. Bayesian Inference). Causal modeling is still relatively new, and is not yet as widely used as it could be. An open problem is how to specify what the model of the world looks like from data (rather than just reasoning over a model that is given). If this problem is solved, we could see major improvements in the ability to provide analyses like those you ask about.
Can AI provide a better understanding about whether the runaway scenario described by some climatologists is realistic?
Probably not. Predicting future events without any observable precedents is not something that anyone can be sure about. The questions raised in those kinds of scenarios aren't about calculation, but about modeling. For example, if you believe that there is a large amount of methane trapped in the arctic permafrost, and you believe that temperatures above a certain range release this much faster than in the past, and you believe that methane warms the climate rapidly, then you would tend to believe that the runaway scenarios are plausible. AI can't tell us whether the methane is there (we have to go measure it, though maybe AI can make the measurement more accurate), and can't tell us how temperature will affect the release rate (again, we have models of this, but they rely on different assumptions, and we have to go measure to find out which are right).
Can AI provide better simulations of the impacts of interventions on both climate and the economy, to inform decision making?
Maybe. Agent-based modeling can help with this to some degree, and is arguably part of AI. In fact, it already has been to some degree, by Beeger & Troost in J. Agriculture Economics, in 2014 and again in 2017, though interest in this kind of modeling looks like a pretty new development in this area. Although ABM can give us reasonable models and help simulate the impact of interventions, ultimately they are just one modeling tool among many. Their potency may improve if more realistic agent models are used, but it is not clear that AI is going to provide advances in this area in the near future.
Automation of climate planning so that the harmful effects of the irresponsibility of one geopolitical entity wishing to industrialize without constraint on other geopolitical entities can be mitigated
Probably not. Although AI techniques have made some kinds of economic planning problems a lot easier, the main barrier to the effects you describe is a social/political one: countries are sovereign, and the world operates as a de facto anarchy (i.e. the UN is impotent). Your AI model can tell, say, India not to industrialize, but Indians want to enjoy the same kind of lifestyle improvements that Americans do, and would rather enjoy them sooner than later. India would collectively rather that Americans put an enormous tax on their carbon emissions, drive less, eat much, much less beef, and stop flying everywhere, than that Indians continue living on the equivalent of $7,000 each per year. In contrast, Americans would rather that Indians just wait a few decades while the developed world decarbonizes, and only industrialize once we have enough solar panels to replace all our current needs within current industrial economies.
Basically this is a resource allocation problem within an anarchy: we can only burn X carbon within Y time, everyone wants to burn some, and the only way to enforce contracts is with the threat of massive violence (or massive economic sanctions, which in turn, are imposed through the threat of violence against other actors if they trade with the sanctioned country). AI can help us answer questions like "How much should the USA pay India in exchange for India not industrializing this year?", see, e.g. work in Auction Theory, but AI can't actually make nations do those things if they can't reach a diplomatic compromise.


What is the correct name for state explosion from sensor discretization? The position of a robot on a map contains of an x/y value, for example $position(x=100.23,y=400.78)$. The internal representation of the variable is a 32bit float which is equal to 4 byte in the RAM memory. For storing the absolute position of the robot (x,y) only $4+4=8$ bytes are needed. During the robot movements, the position is updated continuously.

The problem is, that a 32 bit float variable creates a state space of $2^{32}=4294967296$. Which means there are endless amount of possible positions in which the robot can be. A robot control system maps the sensor readings to an action. If the input space is large, then the control system gets more complicated.

What is the term used in the literature for describing the problem of exploding state space of sensor variables? Can it be handled with discretization?

I would refer to your problem as having a continuous state space.
By using a 32-bit float variable you discritize it. However, creating states for every possible value of a 32-bit float variable is probably too much. You should decide on:

the variable range: what is the real range of the position variables (e.g. from 0 m to 10 m),
and what is the resolution you require for your problem (e.g. 0.01 m or 0.1 m).

Note that you should take into account:

the sensor range resolution,
the required range and resolution for the problem.

Then, based on the number of states you could decide to discretize the states or to use a Monte Carlo approach.

See for example the work of Brechtel et al. (2013).


What are the pros and cons of sparse and dense rewards in reinforcement learning? From what I understand, if the rewards are sparse the agent will have to explore more to get rewards and learn the optimal policy, whereas if the rewards are dense in time, the agent is quickly guided towards its learning goal.
Are the above thoughts correct, and are there any other pros and cons of the two contrasting settings? On a side-note, I feel that the inability to specify rewards that are dense in time is what makes imitation learning useful.

What are the pros and cons of sparse and dense rewards in reinforcement learning?

It is unusual to refer to this difference as "pros and cons" because that term is often used to make comparisons between difference choices. Assuming you have a specific problem to solve, then whether or not the rewards are naturally sparse or dense is not a choice. You cannot say "I want to solve MountainCar, I will use a dense reward setting", because MountainCar has (relatively, for a starting problem) sparse rewards. You can only say "I won't attempt MountainCar, it is too difficult".
In short however, your assessment is correct:

if the rewards are sparse the agent will have to explore more to get rewards and learn the optimal policy, whereas if the rewards are dense in time, the agent is quickly guided towards its learning goal

There is not really any other difference at the top level. Essentially, sparser rewards make for a harder problem to solve. All RL algorithms can cope with sparse rewards to some degree, the whole concept of returns and value backup is designed to deal with sparseness at a theoretical level. In practical terms however, it may take some algorithms an unreasonable amount of time to determine a good policy beyond certain levels of sparseness.

On a side-note, I feel that the inability to specify rewards that are dense in time is what makes imitation learning useful.

Imitation learning is one of many techniques available to work around or deal with problems that have sparse reward structure. Others include:

Reward shaping, which attempts to convert a sparse reward scheme to a dense one using domain knowledge of the researcher.

Eligibility traces, which back up individual TD errors across multiple time steps.

Prioritised sweeping, which focuses updates on "surprising" reward data.

Action selection planning algorithms that look ahead from the current state.

"Curiousity" driven reinforcement learning that guides exploration to new state spaces independently of any reward signal.


How do we define intention if there is no free will? There is an idea that intentionality may be a requirement of true intelligence, here defined as human intelligence.

But all I know for certain is that we have the appearance of free will.  Under the assumption that the universe is purely deterministic, what do we mean by intention?

(This seems an important question given that intention is not just a philosophical matter in relation to definitions of AI, but involves ethics in the sense of application of AI, "offloading responsibility to agents that cannot be meaningfully punished" as an example.  Also touches on goals, implied by intention, whether awareness is a requirement, and what constitutes awareness. I'm interested in all angles, but was inspired by the question "does true art require intention, and, if so, is that the sole domain of humans?")

My thoughts.
The short answer is: you can't.
The long answer is that since we're searching for a new definition of a term when removing a necessary (in my opinion) precondition for it to exist, the question becomes "can you make up a new definition for what you intuitively and empirically understand as intention, while removing free will from the picture?". I'm going to give it a shot.
First of all, there's a lot to be said about whether or not the idea or intuition of intention even exists in our collective discourse based on the latent assumption that free will is a thing. As in, before any rigorous definition, even the intuitions encoded in discourse, philosophy, art, and other languages as "intention" could very well be as invalid as the assumption of free will itself.
That being said, I'm a fan of Deleuze's model for people and other entities as machines of input, output and internal state (not his wording, but I paraphrase and in consequence, interpret and alter for the purposes of my point). It's not perfect, but I run to it a lot to answer these question as I find it very refreshing, often lacking in bias and having good explanatory power compared to the usual romance-foo that dominates these conversations. If that's the case you could pretty much define intention not as a self-started force but as a product of a much blurrier mechanism, namely the non-deterministic characteristic of this rhizomatic soup. Whether or not an input or output will exist, what kind will it be, what internal state will it find or cause in the machine and the long term dependencies between these interactions seem to me like a convincing enough candidate for the cause of any intuition (or illusion if you like a more cynic vocabulary) of "intention". It's pretty much an emergent symbol we use, assuming the form of a force for the setup and function of new connections, that will in their complication or pure non-determinism spawn even more intention in the network.
tl;dr: Intent could be the most basic expression of the RNG of the universe.


Does net with ReLU not learn when output < 0? The derivative of ReLU is 0 if its output is lower than 0 -  $d ReLU(x)/dReLU$ is $0$ if $x < 0$. Let's denote some net's output by $Out$, so if this net's last layer is ReLU then we get that $dOut/dReLU$ is $0$ if $Out < 0$. Subsequently, for every parameter $p$ in the net we would get that $dOut/dp$ is $0$. Does that mean that for every sample $x$ such that $Out(x) < 0$ the net doesn't learn at all from that sample since the derivative for each parameter is $0$?

There is no benefit to using ReLU as the output activation of a neural network. As you said, the network will ignore training labels below zero and it will train on labels above zero as if no output activation were present. However, the problem you're describing can also occur for individual units of hidden layers, where ReLU activations are common. This is known as the dead ReLU problem. In practice, it is rarely a problem but it can be avoided with smooth rectifiers like ELU and Swish. Another interesting ideas is CRelu, which concatenates both positive and negative parts of the pre-activation, resulting in twice as many outputs, half of which always receive a non-zero gradient.


Is an optimization algorithm equivalent to a neural network? Is an optimization algorithm equivalent to a neural network?

Recurrent neural networks (RNNs) are Turing complete (that is, they can simulate any Turing machine), which means that, in theory, they can execute any algorithm that a Turing machine can, so it can also run any optimization algorithm that can be run on a Turing machine (e.g. gradient descent). See this good answer to the question Meaning (and proof) of "RNN can approximate any algorithm" for a more exhaustive discussion of this fact.

A neural network is a model: it represents some (fixed) function, given some fixed weights (which can be thought of as the state of the NN). However, as opposed to most optimization algorithms, the weights or parameters of a NN can be changed by an optimization algorithm (like gradient descent), but this will arguably no more be the same NN. So, in theory, you could have an RNN that approximates gradient descent and back-propagation, so that a neural network trains another neural network.

An optimization algorithm is usually not trained to approximate any continuous function. It is just a possibly parametrized program that runs on a Turing machine. However, note again that a specific optimization algorithm corresponds to a specific NN, so, in this sense, a specific optimization algorithm is equivalent to some NN.


Neural Network for Optical Mark Recognition? I've created a neural net using the ConvNetSharp library which has 3 fully connected hidden layers. The first having 35 neurons and the other two having 25 neurons each, each layer with a ReLU layer as the activation function layer.

I'm using this network for image classification - kinda. Basically it takes inputs as raw grayscale pixel values of the input image and guesses an output. I used stochastic gradient descent for the training of the model and a learning rate of 0.01. The input image is a row or column of OMR "bubbles" and the network has to guess which of the "bubble" is marked i.e filled and show the index of that bubble.

I think it is because it's very hard for the network to recognize the single filled bubble among many.

Here is an example image of OMR sections:

Using image-preprocessing The network is given a single row or column of the above image to evaluate the marked one.

Here is an example of a preprocessed image which the network sees:

Here is an example of a marked input:

I've tried to use Convolutional networks but I'm not able to get them working with this.

Basically, my question is that what type of neural network and network architecture should I use for this kind of task?. An example of such a network with code would be greatly appreciated.

I have tried many preprocessing techniques such as background subtraction using the AbsDiff function in EmguCv and also using the MOG2 Algorithm and I've also tried threshold binary function but there still remains enough noise in the images which makes it difficult for the neural net to learn.

I think this problem is not specific to using neural nets for OMR but for others too. It would be great if there could be a solution out there that could store a background/template using a camera and then when the camera sees that image again, it perspective transforms it to match exactly to the template

I'm able to achieve this much - and then find their difference or do some kind of preprocessing so that a neural net could learn from it. If this is not quite possible, then is there a type of neural network out there which could detect very small features from an image and learn from it.  I have tried Convolutional Neural Network but that also isn't working very well or I'm not applying them efficiently.

From what I understand, don't bother with a CNN, you have essentially perfectly structured images.

You can hand code detectors to measure how much filled in a circle is.

Basically do template alignment and then search over the circles.

Ex a simple detector would measure the average blackness of the circle which you could then threshold.


How to connect AI neural network processor to laptop? I have an average laptop.

How can I connect specialized AI neural network processors (say, Intel Nvidia or Intel Nervana https://venturebeat.com/2018/05/23/intel-unveils-nervana-neural-net-l-1000-for-accelerated-ai-training/) to thelaptop.
Should I buy some external motherboard or even server unit with NN processors inside or is there available more lightweight solution like external HDD?

Check of deep learning box on the internet.
You will have to think hard in order to make a good and cost effective one.

But there are a few people who have made one of their own and shared their experiences.

Here are some medium articles:

Building your own Deep Learning dream machine

Building a 270* Teraflops Deep Learning Box for Under $10,000


Convert input dataset given in hex addresses to int I have created an LSTM Neural Network which take as input the following format in an .csv file

sinewave
0.841470985
0.873736397
0.90255357
0.927808777
0.949402346
0.967249058
0.98127848
0.991435244

How can I write some code so it can take as input hex addresses and convert them to int ?

eg the following .xlsx file containing 400.000 samples

0xbfb22b18
0xbfb22b14
0xbfb22b10
0xbfb22b0c
0xbfb22b18
0xbfb22b14
0xbfb22b10
0xbfb22b0c
0xbfb22b18
0xbfb22b14
0xbfb22b10
0xbfb22b0c

Add train_data 1st

import pandas as pd

df = pd.read_excel('data.xlsx', index_col=None, names=['train_data'])
df['train_data'] = df['train_data'].apply(lambda x: int(x, 16))
df.to_csv('data.csv', index=False)


What introductory books to reinforcement learning do you know, and how do they approach this topic? Currently, I'm only going through these two books

Reinforcement Learning: An Introduction, by Sutton and Barto: RL explained on an engineering level (mathematical, but readable for a non-mathematician). Elementary notions from probability and statistics are required (conditional probability, total probability theorem, total expectation theorem, and similar. The MIT RES.6-012 "Introduction to Probability" course is a great source of information for these topics.).

Deep Reinforcement Learning, by Miguel Morales: this book introduces the main elements of reinforcement learning in a less formal way than Sutton and Barto (derivations for some equations are not given), using examples to describe the math.

What other introductory books to reinforcement learning do you know, and how do they approach this topic?

In addition to the ones you mentioned, I would add Algorithms of Reinforcement Learning by Csaba Szepesvári. There is a number of professors who use it as a reference in their RL teaching materials (for example this one).
It generally follows the same outline as Sutton & Barto's book (except the part on bandits, it is included in the Chapter on Control). In fact, it may be considered as a condensed version of Sutton & Barto (about 100 pages). In addition, it's freely available online.
I like the author's justification as to why he wrote this book, so I'm just going to quote it:

Why did I write this book? Good question! There exist a good number of
really great books on Reinforcement Learning. So why a new book? I had
selfish reasons: I wanted a short book, which nevertheless contained
the major ideas underlying state-of-the-art RL algorithms (back in
2010), a discussion of their relative strengths and weaknesses, with
hints on what is known (and not known, but would be good to know)
about these algorithms.


How can I predict the true label for data with incomplete features based on the trained model with data with more features? Suppose I have a model that was trained with a dataset that contains the features (f1, f2, f3, f4, f5, f6). However, my test dataset does not contain all features of the training dataset, but only (f1, f2, f3). How can I predict the true label of the entries of this test dataset without all features?

Assuming that you have access to the training data set, you could use an autoencoder network to predict what features f4, f5, f6 'could be' for the test data set. The way to do this is to train the autoencoder on the training data set with features f1, f2, f3 as inputs, and then use f1,f2,f3,f4,f5,f6 as the output of the network. The autoencoder then effectively learns to map any input samples with (f1,f2,f3) to (f1,f2,f3,f4,f5,f6). By passing your test data through the autoencoder, you can then use the output and pass it to your model.


What does it mean to do multi-dimensional processing with tensors in tensor cores? In some tweets about NeurIPS 2018, this video from NVIDIA appeared. At around 0.37, she says:

  If you think about the current computations in our deep learning systems, they are all based on Linear Algebra. Can we come up with better paradigms to do multi-dimensional processing? can we do truly tensor-algebraic techniques in our tensor cores?

I was wondering what she is talking about. I'm not an expert so I'd like to understand better this specific point.

Both PR and Theoretically Valid

Although Anima Anandkumar's presentation is a puff piece for NVidia, her representation is not contrary to theory.

  ... next level [above NVidia's GPU success] ... means new algorithmic research. So, if you think about the current computations in our deep learning systems, they are all based on linear algebra. Can we come up with better paradigms to do multi-dimensional processing. Can we do truly tensor algebraic techniques in our tensor cores, and what kind of new architectures will this realize?

Tensors

A tensor is the extension of the idea of functions and vectors. Vectors emerged allow the encapsulation of multiple variables of the same type into a single unit to express multidimensional properties such as position or force. Variables that are not encapsulations of multiple variables are then referred to as scalars. When such multidimensional properties are compositions of functions rather than variables because their values are dependent upon other variables, they are vector fields.

To more cleanly model electromagnetism and gravity in mathematics, higher levels of variable and functional encapsulation were required. Scalar variables and functions that return scalar variables are thus tensors of rank one (1). Vectors and vector fields are tensors of rank two (2), and higher ranks are extensions of the pattern created by scalars and vectors. A matrix containing functions can be represented as a tensor of rank three (3), encapsulating its elements into a single unit.

Digital Processing and Integration Trend

CUDA cores, which are the signal processing units used in current generation NVidia GPUs (graphical processing units) can be used for production of two or three dimensional rendering or leveraged to produce parallel processing of signals through an artificial network. This is following the trend to delegate to digital circuitry the most significant bottlenecks in algorithm execution. VLSI (very large scale integration) technology is the logical result of this trend.

Industrial and military calculators with tests and branching running on rows of racks of relays
Add speed with tubes and load programs with punch cards
Reduce power and improve reliability and speed with transistors, magnetic core memory, and paper tape
Integrated circuits on motherboards with vinyl tape
Handheld programmable calculators for students
Microprocessors with magnetic disks
Floating point, digital signal, and graphics processors to extend microprocessors
Multiple cores (repeating large scale circuit patterns on a single substrate)
Re-purposing and extending graphics rendering circuits to offload artificial network computations to GPUs

With all these, speed, size, energy thrift, and convenience are not really a shift in the original paradigm that started with Norbert Wiener, Claude Shannon, Alan Turing, John von Neumann, and others. In fact, computing is still catching up in several ways with the perspectives of these pioneers and a long way off from producing in VLSI common concepts in science. Software is the solution, which is why it is called soft, meaning flexible, not necessarily weak. However, flexibility sacrifices speed and capability, thus the trend above.

The Parallel Computing Challenge

Part of what Anima Anandkumar is stating that the signal paths in current VLSI processors are still a much lower level of abstraction than the ideas of mathematicians, physicists, and AI engineers. Scientific theory describes probability, statistical distribution, expectation, force, loss, gain, pain, reward, memory, momentum, semantics, combination, and correlation at a much higher level than current digital circuitry.

The use of sequential algorithms is the splaying out in time what could be a massively parallel operation. The serial algorithm limits the rate of processing. The finding of ways of dealing with things in parallel in mathematics can be done with a pencil. In computers, paralleling algorithms and finding parallel processing structures in VLSI form that are as flexible as software in some respects is much more challenging and thus far behind.

This is one problem that has been the focus of research in at least twenty large corporate and government labs since for half a century and has been the intention of VLSI from the onset. The work is not specific to NVidia. It is not a new problem and the approach to solving it has been along this paradigm.

Add abstraction and encapsulation in the mind.
Express it in mathematics.
Write it as a serial algorithm, leveraging whatever parallel constructs are supported by programming languages and libraries that can leverage VLSI parallelism or computing clusters.

The last seventy years of computing machinery development has been bringing hardware, operating systems, and software closer to the level of mathematical expressions that were twenty to two hundred years old. That may change, and everyone wants to ride the new wave.

Anima Anandkumar and her counterparts in IBM, Intel, Google, Microsoft, the U.S. Navy, Amazon, Alibaba, and the other corporate and government laboratories do not state (because it is either classified or company confidential) what they intend to do to further the parallelization of computing. Whatever they do along those particular lines would not be a paradigm shift but rather a next step along the current paradigm.

Enters Corporate Strategy

They would also not state what they may be doing that is not in that paradigm. They are constrained to only give hints without theoretical substance. If they were working on a chip that exhibited what the human brain exhibits when neurons grow and connect according to DNA based propensity, they wouldn't say that in a technically precise way. Corporate secrecy is part of the global economic game play, not tipping the hand. They have their game face on.

The idea of moving from linear to non-linear is a good public relations theme and when used for puff pieces is not conclusively technical. The pitch usually goes along this line.

  What they did was very linear. We are moving into a non-linear space.

It is an attempt to claim that what was done was primitive and the impending game changing advancement is coming from the speaker and their people. Sometimes it works to create a temporary hike in the value of tradable securities, which that company needs as of this writing. If there is a true change in the game, it will be known when it is released. Those who have worked in laboratories for years know to wait until something is released that demonstrates, when the example code is configured and ran, what game change actually occurred, if any. Or they develop the game changer themselves, which is why it is a lab.

Ambiguity of 'Linear' Even in Mathematics

Also note that curved lines are still lines and the term linear can mean two things depending on the context.

Conformant to the linear equation $\vec{Y} = V \vec{X}$, such that it graphs as a line, plane, or higher dimensional flat surface, with constant gradient and without curvature.
Conformant to the principles of linear algebra, which would include spaces, eigenvalues, regression with high-order polynomials, and a number of constructs that involve varying gradient and curvature.

Predicting Paradigm Shifts

The skepticism developed after watching tech company PR for some time does not necessarily dismiss game changing technology advancements of the past and potentials of the future. Any open source project team, individual, corporation, or government lab might do something that shifts the paradigm, usually over a period of years. Classic examples:

Rope
Wheel
$F = m a$
Oxygen (that air is not an element)
Electromagnetism
Alternating current energy transmission
Relativity
Internal combustion
Radio

Computing examples:

Information theory
Cybernetics
First transistor
LISP and FORTRAN
C and UNIX
TCP/IP

Shifts do not need to be as far reaching and game changing as these to have an impact. Who will achieve the next with regard to AI concepts, information structure, algorithm, execution environment is not something very easy to predict. Consider those that lived before any of the above and try to imagine them trying to predict that the projects of Isaac Newton, Antoine Lavoisier, Michael Faraday, Nicola Tesla, Norbert Wiener, Claude Shannon, Ken Thompson, Dennis Ritchie, or any of the others were the seeds of the next paradigm shift.

There is research into analog artificial networks, neuromorphic hardware, semantic modelling, graph algorithms, and other potential game changers, each of which has impressive conceptual foundations and is discussed in some of the Q&A here. These are a few.

What topologies are largely unexplored in machine learning?
Novelty Search Mutation Algorithm
Can deep networks be trained to prove theorems?
If digital values are mere estimates, why not return to analog for AI?
Are commercially available neural ICs digital?

What of these may point to the beginnings of paradigm shift cannot be known, and even if an idea first posted here or referenced from here is the seed, it may not be known later. The multilayer perceptron may be the seed of future streets and highways being dominated by automated vehicles in 2090, but no one in a million people will realize in seventy years that the trend toward AV research was seeded by MLP enthusiasm from the first decade of this century.

Gaps to be Filled and NVidia is a Contendere

All this aside, we use NVidia hardware every day for robotics and analysis, so they have credibility resulting from past success. If they produced a chip that does something remarkably clever before Intel or the IBM-MIT collaboration, it would be a small surprise, but not it is not a completely unbelievable possibility. Certainly the comprehension of Hilbert spaces, semantics, and topology are limited in the computer science field, and a paradigm shift toward greater comprehension of them or some new thing that is not even part of mathematical thought today would add some needed diversity to the computing industry.