Does Monte Carlo Tree Search not work on games without the same initial state? I'm curious how you would apply Monte Carlo Tree Search to a game that has a random initial state. You generate a tree where the root node is the initial state, then you expand if the options from that state are not explored yet.
I'm also wondering how this works in 2 player games. After your opponent moves, does each state in the tree have a key to look up in a dictionary? Otherwise, the algorithm won't know what to do when there's a jump in a state between choosing your action on your turn and when your opponent moves, unless you also store your opponent's move in the tree.

If the initial state is not always the same, but if your agent is allowed to observe what the initial state is before it has to start running the search algorithm, there's basically no problem; it has all the information it needs when it starts running the tree search. This is how we typically use MCTS (or any other tree searches): we first observe what the current state looks like, and then start running the tree search for this state.
If for whatever reason you already have to start running your tree search without being allowed to observe which initial state has been randomly drawn, you can easily just pretend that you actually do have a "dummy", deterministic initial state which you can observe right before the real initial state, and pretend that the random sampling of an initial state is actually a random event / random transition resulting from a "dummy action". Then you can handle this scenario in exactly the same way that you would any normal game that has non-deterministic transitions (such as games that involve dice): if you have access to explicit knowledge of which stochastic events are possible, and what their probabilities are, you can encode them as chance nodes. If you do not have such explicit knowledge available, you can use an "open-loop" MCTS. See also: https://ai.stackexchange.com/a/13919/1641

Presence of opponents / other players in the game is a different issue. MCTS and other game tree search algorithms were designed specifically for this, they have no problems handling that. In a tree search, you do not just have nodes for your own agent and the states in which it is allowed to act; you also have nodes for any opponents and any states in which they are allowed to act, and the tree search enables you to reason about what actions your opponents are likely to take.


Validation Accuracy remains constant while training VGG? I posted this question on stackoverflow and got downvoted for unmentioned reason, so I'll repost it here, hoping to get some insights
This is the plot

This is the code:
with strategy.scope():

  model2 = tf.keras.applications.VGG16(
    include_top=True,
    weights=None,
    input_tensor=None,
    input_shape=(32, 32, 3),
    pooling=None,
    classes=10,
    classifier_activation="relu",
  )

  model2.compile(optimizer='adam',
                loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
                metrics=['accuracy'])

  history = model2.fit(
            train_images, train_labels,epochs=10,
            validation_data=(test_images, test_labels)
            )

I'm trying to train VGG16 from scratch, hence not importing their weights I also tried a model which I created myself, with same hyperparameters, and that worked fine
Any help is highly appreciated
Heres the full code

Ok, I solved this problem
The simple thing was that learning rate was too big
I changed the code to this
LR = batch_size/((z+1)*100000)
LR=LR/3

instead of
LR = batch_size/((z+1)*1000)
LR=LR/3

and it seems to work well


How can we compute the ratio between the distributions if we don't know one of the distributions? Here is my understanding of importance sampling. If we have two distributions $p(x)$ and $q(x)$, where we have a way of sampling from $p(x)$ but not from $q(x)$, but we want to compute the expectation wrt $q(x)$, then we use importance sampling.

The formula goes as follows:

$$
E_q[x] = E_p\Big[x\frac{q(x)}{p(x)}\Big]
$$

The only limitation is that we need a way to compute the ratio. Now, here is what I don't understand. Without knowing the density function $q(x)$, how can we compute the ratio $\frac{q(x)}{p(x)}$?

Because if we know $q(x)$, then we can compute the expectation directly.

I am sure I am missing something here, but I am not sure what. Can someone help me understand this?

The rationale behind importance sampling is that $q(x)$ is difficult to sample from but easy to evaluate. Or at least you can easily evaluate some $\tilde{q}$ such that:
$$
\tilde{q}(z) = Zq(z)
$$
where $Z$ (scalar) might be unknown. The geometrical example would be here e.g. sampling uniformly from an area under the curve $q(x)$ (in general it's not easy).

  Because if we know $q(x)$, then we can compute the expectation directly.

That's the task we're trying to solve to begin with. And calculating expectation might be hard if we can't sample efficiently from $q$.

Say you want to compute an expectation of $x$, $E[x]$. For this you need to calculate the following integral:
$$
E[x] = \int{xq(x)dx}
$$
where $q$ is a probability distribution of $x$ for which you have an expression - so you can evaluate $q(x)$ (up to the constant of proportionality). This integral might be hard to evaluate analytically so we need to use other methods such as Monte Carlo. Let's say it is hard to generate samples from $q$ (as per example above, e.g. generating samples from the area under the curve $q(x)$ uniformly).

What you can do is to calculate an expectation under a simple distribution $p$ (proposal distribution) which is a distribution of your choice that needs to allow you to easily sample from it (say Gaussian). Then you can rewrite your integral as:
$$
E_q[x] = \int{xq(x)dx} = \int{xq(x) \color{blue}{\frac{p(x)}{p(x)}} dx}
= \int{x \frac{q(x)}{\color{blue}{p(x)}} \color{blue}{p(x)} dx}
= E_p \Big[{x\frac{q(x)}{p(x)}}\Big]
$$
(added index $p$ and $q$ to expectation to denote the sampling distribution). Now you can approximate the last expectation by Monte Carlo:
$$
E_p \Big[{x\frac{q(x)}{p(x)}}\Big] = \frac{1}{S} \sum_{s}{x^{(s)} \frac{q(x^{(s)})}{p(x^{(s)})} }, \ x^{(s)} \sim q(x)
$$


What is 3D face recognition? and how we can check liveness of a face image? Actually what is mean by 3D face recognition? In normal cases we are extracting face encoding s from a 2D image,right?
Is 3D face recognition is used for liveness detection? how its possible?

1. What is typically meant by 3D-face recognition? We are usually extracting the face encoding from 2D-images, right?
Yes. The goal is to reconstruct the three-dimensional shape, as well as the texture of a face from a single or multiple images of that person.
In recent years, "the performance of 2D face recognition algorithms has significantly increased with the use of deep neural networks and the use of large-scale labeled training data." Deep 3D Face Ifentification with the latter still being a notable problem because there are so few of them and they are usually needed to better measure and hence train the deep networks. (The referenced paper gives you a nice overview.) Even with enough data, occlusions, lens distortions or possible variations of expressions, etc. make the problem tough. So many other tricks and augmentation techniques are needed.
2. How can 3D face recognition used for liveness detection?
3D face recognition is used to differentiate 2D spoofs from actual present people. Using multiple time-delayed images one could possibly track if the face moves like a living person as well, or if it is static or stretching weird like a mask, puppet, or moving display.
Nonetheless, often even advanced 2D image-based techniques fall flat in real-world applications because of the used cameras. They might not have enough resolution to recognize if the input is a video of a face on a high-resolution display as explained here.


Do transformers have success in other domains different than NLP? Everybody knows how successful transformers have been in NLP. Is there known work on other domains (e.g that also have a sequential natural way of occurring, such as stock price prediction or other problems)?

When it talks to other domains such as image or music, using transformer will always face a problem of sequence length limitation. To the best of my knowledge, the bottleneck of self-attention which uses a $n^2$ matrix quite limits transformer being applied to other domains. For example, a 32x32 pixel image, means a sequence of 1024 tokens.
OpenAI did some related research, as the followings.
Generative Modeling with Sparse Transformers:
In the paper, transformers with sparse attention are applied to image and waveform.
ImageGPT:
A large transformer model trained on language can generate coherent text, the same exact model trained on pixel sequences can generate coherent image completions and samples. (Abstract from the blog)


Is there anything novel about Zuckerberg's Jarvis? Recently Mark got some attention from the media by stating that he had created Jarvis. Not that I'm against him or anything, but this Jarvis seems to have been done a hundred times before. He's done something which most developers would classify as a home automation system. To me it's more like he did it for the attention. I was kind of taken back by the amount of media attention he got. If you've heard of Jeremy Blum, maybe you may understand what I'm trying to imply here.

I'm just curious as to why he got so much attention. Is there anything technically novel about his system that sets it so much apart from previous ones?

No, there is nothing novel about this system. The main hurdle he had to pass was problems that you will face when your system has a lot of integration points across various APIs provided by different vendors with messy and often outdated documentation.

As far as attention is concerned we live in a world where so called celebrities get attention for anything that they do. Remember that media have only one goal - More and more eyeballs aka more money and nothing else.


How are newer weight initialization techniques better than zero or random initialization? How do newer weight initialization techniques (He, Xavier, etc) improve results over zero or random initialization of weights in a neural network? Is there any mathematical evidence behind this?

There are several ways to answer this question. First of all, there are several mathematical arguments on why using some kind of initialization is better. Consider reading, for example, Xavier et al.. Moreover, there are several numerical experiments showing the importance of initialization.

The motivation for Xavier initialization in Neural Networks is to initialize the weights of the network so that the neuron activation functions are not starting out in saturated or dead regions. In other words, we want to initialize the weights with random values that are not "too small" and not "too large". Thus the purpose is to fix the variance of the input data to each neuron to 1, because this reduces the variance of getting stuck in saturated areas, since the data is in general normalized this is equivalent to fixing the variance of each weight to $1/n$, where n is the number of input weights to the given neuron.


What are the main problems hindering current AI development? I have a background in Computer Engineering and have been working on developing better algorithms to mimic human thought. (One of my favorites is Analogical Modeling as applied to language processing and decision making.) However, the more I research, the more I realize just how complicated AI is.

I have tried to tackle many problems in this field, but sometimes I find that I am reinventing the wheel or am trying to solve a problem that has already been proven to be unsolvable (ie. the halting problem). So, to help in furthering AI, I want to better understand the current obstacles that are hindering our progress in this field.

For example, time and space complexity of some machine learning algorithms is super-polynomial which means that even with fast computers, it can take a while for the program to complete. Even still, some algorithms may be fast on a desktop or other computer while dealing with a small data set, but when increasing the size of the data, the algorithm becomes intractable.

What are other issues currently facing AI development?

we don't really know what intelligence is.
we don't truly understand the best model of intelligence we have available (human intelligence) works.
we're trying to replicate human intelligence (to some extent) on hardware which is quite different from the hardware it runs on in reality.
the human brain (our best model of intelligence) is mostly a black-box to us, and it's difficult to probe/introspect its operation without killing the test subject.  This is, of course, unethical and illegal. So progress in understanding the brain is very slow.

Combine those factors and you can understand why it's difficult to make progress in AI. In many ways, you can argue that we're shooting in the dark. Of course, we have made some progress, so we know we're getting some things right. But without a real comprehensive theory about how AI should/will work, we are reduced to a lot of trial and error and iteration to move forward.


Is RL just a less rigorous version of stochastic approximation theory? After reading some literature on reinforcement learning (RL), it seems that stochastic approximation theory underlies all of it.

There's a lot of substantial and difficult theory in this area requiring measure theory leading to martingales and stochastic approximations.

The standard RL texts at best mention the relevant theorem and then move on.

Is the field of RL is really stochastic approximation theory in disguise? Is RL just a less rigorous version of stochastic approximation theory?

Is the field of RL is really stochastic approximation theory in disguise? Is RL just a less rigorous version of stochastic approximation theory?

No, but reinforcement learning (RL) is based on stochastic approximation theory (SAT), and these two fields overlap.

In RL, you typically assume that the underlying problem can be modeled as a Markov decision process (MDP), and the goal is to find a policy (or value function) that solves this MDP. To find this policy, you can use stochastic approximation algorithms, such as Q-learning, but RL isn't just SAT, where, in general, there isn't necessarily a notion of MDP.

SAT is the study of iterative algorithms to find the extrema of functions by sampling from them and under which conditions these iterative algorithms converge. SAT isn't just applied in RL, but it is applied in many other fields, such as deep learning. The paper Scalable estimation strategies based on stochastic approximations: Classical results and new insights (2015) by P.  Toulis et al. provides an overview of SAT and the connections with other fields (including RL).

To conclude, RL is based on SAT, but RL isn't just stochastic approximation algorithms, so they are distinct fields. If you want to study e.g. the convergence properties of certain RL algorithms, you may need to study SAT. In fact, for example, the typical proof of convergence for tabular Q-learning assumes the Robbins–Monro conditions. However, you can do a lot of RL without even knowing that RL is based on SAT. Similarly, you can do a lot of SAT without ever caring about RL.


Can machine learning help me digest asymmetrical order descriptions? I have order data, here's a sample:

Ninety-six (96) covered pans, desinated mark cutlery.
5 vovered pans by knife co.
(SEE SCHEDULE A FOR NUMBERS). 757 SOUP PANS
115 10-quart capacity pots.
Thirteen (13), 30 mm thick covered pans.

I have over 50k rows of data such as this. In a perfect world, the above would need to be tabulated as such:

count, type
96, covered pan
5, covered pan
757, soup pan
115, pot
13, covered pan

Could machine learning be the correct approach for a problem such as this?

Yes a variant of NLP processing could help find the correct number to extract and type of object in this data.

Compared to the spreadsheet, the raw text data is ambiguous without understanding language to a reasonable depth, and without knowing the business context in order to extract the relevant information.

For instance, you are expecting to extract "soup pan" and "covered pan", but not "capacity pot". Also that parts of phrases such as "30 mm" or "10-quart" are lower importance qualifiers, and not specifying quantity of something.

The current state of the art for extracting this kind of data would be a bidirectional LSTM (a type of Recurrent Neural Network). You would likely get it to flag the parts of each entry that were relevant to the tabulated data you wanted to extract, then feed those into a simpler stage that put them into the spreadsheet. However, there are two caveats:

You need a lot of correctly-labelled training data to get reasonable performance. Using a word embedding layer, such as word2vec or GloVe, should significantly reduce the amount of training data required, but may require a careful pre-processing stage, and may be less useful when you have a lot of jargon in your data.
Performance is never perfect, and the system can still make stupid mistakes, because it does not truly understand the text is is dealing with. That applies to all ML approaches to this problem, and likely also to coding up an "expert system", although it may be easier to write the expert system to recognise when it had failed and ask for help.


How can I learn tensors for deep learning? I've seen in most deep learning papers use tensors. I understood what tensors are, but I want to dive into them, because I think that might be beneficial for further studies in Artificial Intelligence. Do you have any suggestion (e.g. books or papers) about that?

In deep learning (and, in general, machine learning), tensors are multi-dimensional arrays. You can perform some operations on these multi-dimensional arrays (depending also on the specific implementations and libraries). These operations are similar to the operations you can apply to vectors or matrices, which are just specific examples of multi-dimensional arrays. Examples of these operations are

indexing and slicing (if you are familiar with Python, these terms should not scare you)
algebraic operations (such as multiplication of a tensor with another tensor, which includes numbers, vectors or matrices), which typically support broadcasting
reshaping (i.e. change the shape of the tensor)
conversion to or from another format (e.g. a string)

TensorFlow provides an article that discusses these tensors, so I suggest that you read it.
In mathematics, tensors are not just multi-dimensional arrays. They are multi-dimensional arrays that need to satisfy certain properties (in the same way that matrices need to satisfy certain properties to be called matrices) and are equipped with certain operations.
The paper A Survey on Tensor Techniques and Applications in Machine Learning (2019) Yuwang Ji et al., published by IEEE, provides a comprehensive overview of tensors in mathematics. It is full of diagrams that illustrate the concepts and the explanations are concise. Some of the explanations in this paper may not be very useful to develop deep learning applications, but some of the illustrations and explanations (especially, in the first pages, which are the only ones that I read) will give you some intuition behind the tensors or multi-dimensional arrays used in deep learning).
So, tensors in deep learning (DL) may not be exactly equivalent to the tensor objects in mathematics, because they may not satisfy all the required properties or some of the operations in the specific libraries may not be implemented, but it is fine to call them tensors because a tensor in mathematics is also a multi-dimensional array, to which you can apply operations (some of them are implemented in the DL libraries).


How do I keep my system (online) learning if I can get ground truth labels only for examples flagged positive? I have a binary classifier (think of it as a content moderation system) that is deployed after having being trained via batch learning.

Once deployed, humans review and check for correctness only items predicted positive by the algorithm.

In other words, once in production if I group predictions of the model on unseen examples in the confusion matrix

+-----------+-----------------+
|           |   Ground-truth  |
|           +-----+-----------+
|           |     | Neg | Pos |
+-----------+-----+-----+-----+
|           | Neg | x11 | x12 |
| Predicted +-----+-----+-----+
|           | Pos | x21 | x22 |
+-----------+-----+-----+-----+

I have access to all the ground-truth labels of the elements counted in $x_{21}$, $x_{22}$ (the predicted-positive)
I know the sum of $x_{11}$ and $x_{12}$, but not their values
I do not have access to the ground-truth labels of the elements predicted-negative.

This (suboptimal) setup allows to measure precision $\frac{x_{22}}{x_{21} + x_{22}}$, while recall stays unknown as elements predicted negative are not examined at all (ground-truth labels of negatives can't be assigned due to resource constraints).

Information gathered from users about the (true and false) positive elements can be used to feed a retraining loop... but

are there any "smart" learning recipes that are expected to make the algorithm improve its overall performance (say, the F1 score for the positive class) in this setting?
what's a meaningful metric to monitor to ensure that the performance of the model is not degrading?* (given the constraint specified here, F1 score is unknown).

Thanks for any hint on how to deal with this!

 * One solution could be to continuously monitor the F1 score on a labeled evaluation set, but maybe there's more one can do?

A first question that I think is important to consider is: do you expect the data that you're dealing with to be changing over time (i.e. do you expect there to be concept drift)? This could be any kind of change. Simply changes in how frequent certain inputs are, changes in how frequent positives/negatives are, or even changes in relations between inputs and ground-truth positive/negative labels.

If you do not expect there to be concept drift, I'd almost consider suggesting that you may not have that big of a problem. It might be worth not doing anything at all with the data you receive online, and just stick to what you learned initially from offline data. Or you could try to use those few extra predicted-positive samples that you get for finetuning. You'd just have to be careful not too change your model too much based on this, because you know that you're not receiving a representative sample of all the data anymore here, so you might bias your model if you pay too much attention to only this online data relative to the offline data.

I guess the question becomes much more interesting if you do expect there to be concept drift, and it also seems likely that you are indeed dealing with this in most of the situations that would match the problem description. In this case, you will indeed want to make good use of the new data that you get online, because it can allow you to adapt to changes in the data that you're dealing with.

So, one "solution" could be to just... ignore the problem that you're only learning online from a biased sample of all your data (only from the predicted-positives), and just learn anyway. This might actually not perform too badly. Unless your model is really incredibly good already, you'll likely still get false positives, and so also still be able to learn from some of those -- you're not learning exclusively from positives. Still, the false positives won't be representative of all the negatives, so you still have bias.

The only better solution I can think of is relaxing this assumption:

  Once deployed, humans review and check for correctness only items predicted positive by the algorithm.

You can still have the humans focus on predicted positives, but maybe have them inspect a predicted-negative also sometimes. Not often, just a few times. You can think of this as doing exploration like you would in reinforcement learning settings. You could do it randomly (randomly pick predicted negatives with some small probability), but you could also be smarter about it and explicitly target exploration of instances that your model is "unsure" about, or instances that are unlike data you've seen before (to specifically target concept drift).

I have a paper about something very similar to this right here: Adapting to Concept Drift in Credit Card Transaction Data Streams Using Contextual Bandits and Decision Trees. Here the assumption is that we're dealing with (potentially fraudulent) transactions, of which we can pick out and manually inspect a very small sample online. The only real difference in this paper is that we assumed that different transactions also had different monetary "rewards" for getting correctly caught as positives, based on the transaction amount. So a transaction of a very high amount could be worth inspecting even if we predicted a low probability of being fraudulent, whereas a transaction of a very low amount might be ignored even if it had a higher predicted probability of being fraudulent.

  what's a meaningful metric to monitor to ensure that the performance of the model is not degrading?* (given the constraint specified here, F1 score is unknown).

Having a labelled evaluation set for this could be useful if possible... but it also might not be representative if concept drift is expected to be a major issue in your problem setting (because I suppose that the concept drift that you deal with online would not be reflected in an older, labelled evaluation set).

Just keeping track of things that you can measure online, like precision, and how it changes over time, could be useful enough already. With some additional assumptions, you could get rough estimates of other metrics. For instance, if you assume that the ratio $\frac{TP + FN}{FP + TN}$ between ground-truth-positives and ground-truth-negatives remains constant (remains the same as it was in your offline, labelled data), you could also try to extrapolate approximately how many positives you've missed out on. If your precision is dropping over time (your true positives are getting lower), you know -- assuming that fraction remains constant -- that your false negatives somewhere else in the dataset must be growing by approximately the same absolute number.


Why is this deep Q agent constantly learning just one action? I'm trying to implement deep q learning in the OpenAI's gym "Taxi-v3" environment. But my agent only learns to do one action in every state. What am I doing wrong? Here is the Github repository with the code.

I thought about my input-layer. I had the 500 states one hot encoded. So 499 of every input node would be 0. And 0 is very bad in an neural network. I tried the same code with the "CardPole-v0" and it worked.

So think about your input guys


Using ML to encypher data for production I am looking for research and experience working with ML models to ingest data for tasks, like text analysis, and creates a system that copies (or in other words enciphers) the input data, to then reproduce it in the future without the original.

I'm interested in how ML models can be used in this way to obfuscate information without too much information loss by the model, e.g. overfitting on purpose to create a new representation of the input information.

It sounds like you are trying to compress data, and then recover the same data later.

The most common tool for this task is an autoencoder. This model accepts data as input, and then learns to compress it and decompress it to produce something as close as possible to the original data. By making the middle layer of an autoencoder narrower, you can make the compression more lossy. By making it wider, you can make it less lossy.


Which local minima to choose according to the shape of the error surface? The following plot shows error function output based on system weights.
Two equal local minima are shown in green pointers. Note that the red dots are not related to the question.

Considering the amount of convex in the local minima, is there any way to opt between these two local minima?

Just sharing my thoughts on this:

A local minimum in the loss function is just a synthetic measure expressing how well the model is performing on the target dataset (I'm assuming it's the test set)
Letting aside the absolute values, you could also consider the steepness of that region as follows:

the first LM is at the bottom of steep region which means the related parametrization is quite unstable as if you change it slightly the performance drop quickly (loss increases quickly) so you could interpret this
as a parametrization with not great generalization capability (it is not robust against small changes): always remember that what you are actually looking for is model able to generalize well when in production (i.e. after you have deployed it so when training, validation, test is finished) and test set is typically a small proxy for the data observed in production (but it really depends on the final application itself)
the second LM is related to a much less steep region: even if you change the parametrization slightly you do not observe a big performance change which could be interpreted as a more stable parametrization hence with better possibility to generalize on unseen data


What are the possible social consequences of training neural networks with artificially generated data? Machine learning models and, in particular, neural networks are trained with data often collected from the real world, such as images of real people.
Meanwhile, neural networks (such as GANs) are also used for data generation. Each year, they become better at this task to the point that even humans are not able to distinguish real-world data from the artificially generated one.
So it is possible that neural networks will start to learn with data that was generated by other neural networks, because it will look as real even for a human, but naturally will be not related to the real world.

Will it lead to some machine learning collapse?

Might it lead to some changes in human's perception of the world, because people get a very big part of their knowledge using computers, connected to the Internet?

Is anyone thinking about this potential problem?

We can already observe information bubbles on social media, where the circle is that the ML algorithms learn what content people like and give more similar content based on clicks and so on. From a single wrong click, you could enter a bubble and never come out if you don't take care or be aware.
This happens with humans, so the same may apply to computers. Checking backgrounds, like click saver for click titles as an ultimate example, would save the machines from circles too. Information is not different whether it is made in consecutive rounds by humans or machines.
For example, if you make a thesis, not necessarily all things you write are your own investigation and research. You use quotes, refer to papers, and so on. For information sources, you have to be careful if the source is respected and considered to be meaningful and correct.
The same carefulness needs to be applied to machine-generated content. To the responsible - the last responsibility is with the reader to believe or not what you're reading.


How would we know if AI development will continue to be multipolar, or will become monolithic? Currently, many different organizations do cutting-edge AI research, and some innovations are shared freely (at a time lag) while others are kept private. I'm referring to this state of affairs as 'multipolar,' where instead of there being one world leader that's far ahead of everyone else, there are many competitors who can be mentioned in the same breath. (There's not only one academic center of AI research worth mentioning, there might be particularly hot companies but there's not only one worth mentioning, and so on.)

But we could imagine instead there being one institution that mattered when it comes to AI (be it a company, a university, a research group, or a non-profit). This is what I'm referring to as "monolithic." Maybe they have access to tools and resources no one else has access to, maybe they attract the best and brightest in a way that gives them an unsurmountable competitive edge, maybe returns to research compound in a way that means early edges can't be overcome, maybe they have some sort of government coercion preventing competitors from popping up. (For other industries, network or first-mover effects might be other good examples of why you would expect that industry to be monolithic instead of multipolar.)

It seems like we should be able to use insights from social sciences like economics or organizational design or history of science in order to figure out, if not which path seems more likely, how we would know which path seems more likely.

(For example, we may be able to measure how much returns to research compound, in the sense of one organization coming up with an insight meaning that organization is likely to come up with the next relevant insight, and knowing this number makes it easier to figure out where the boundary between the two trajectories is located.)

Oligopoly vs Monopoly

The terms in the question, multipolar and monolithic, appear to be referring to the micro-economic concepts of oligopoly and monopoly respectively.  Although these concepts are not AI specific, they certainly apply to such development in the way the question suggests.  Leading AI R&D is occurring in  a relatively small number of corporate, governmental, and university research labs.

A simple search for Oligopoly vs Monopoly will cover the differences between these two scenarios sufficiently for all sectors of products and services, and those concepts and observations apply to a large degree in the AI case.

Those who have not studied economics or political science might incorrectly conclude that oligopoly is necessarily related to capitalistic competition and monopoly is related to communism. However, R&D monopolies frequently exist within capitalistic economies and oligopolies frequently exist in a planned economies, such as communist states.

Strengths of Each

Research oligopoly strengths:

Lead to diversity of approaches and target products and services
Lower the probability of ethical, environmental, or social technology abuses because multiple independent teams, even with corporate confidentiality are in place, exert the force of at least a modicum of accountability

Research monopoly strengths:

Efficient in terms of return on research investment for two reasons (1) the division of a fixed amount of available research funding between several labs can increase the R&D required to produce usable results, and (2) unfruitful approaches are more quickly abandoned when their workers can be shifted to other teams within the same organization
Attract all those who have the most talent and experience in the fields needed by the specific R&D into one place to potentially collaborate

Predicting Transitions Between Monopolistic and 'Oligopolistic' Forms

The leading edge of any R&D tends to begin as a monopoly or a 'biopoly'.  Three or more units beginning the same research at the same time or being funded by government at the same time (as in planned economies) is unusual.

Over time, employees tend to leave and start their own organizations, more so in capitalistic societies, but also in planned economies, where government may hear cases to branch off and grant permission or the scientist may, in one way or another, seek relocation to a free economy.

We can see these trends throughout technological history, back to early forms of writing, crop irrigation and plowing, chariot development, and ship building.  Modern examples include the application of the Westinghouse-Tesla model of electrical energy distribution and sales, genetic engineering, speech recognition, and HTTP clients (browsers starting with Mosaic, then Netscape, then IE, then Opera, Chrome, and others).

Unknowns

How far ahead one global AI leader is than another cannot be so easily predicted.  Some companies choose to monetize R&D results sooner in the research cycle than others, depending on their strategy and the diversity of their revenue stream.  For instance, as of this writing, Apple and Facebook can wait a decade to release to the public manifestations of R&D without jeopardizing their financial status.

Furthermore, governments may declassify material (another form of what the question calls time lag) several decades after discovery.  In the case of the Von Neumann model for achieving critical temperature and pressure of fissile or fusion materials, the government may never declassify it.  What the NSA or equivalent organizations outside the United States may decide must fall under an equivalent strategic wall of secrecy cannot be known outside of treasonous disclosure.

The insight in an organization is not sustainable in practice.  Most people do not even know where first insights occurred and two whom they occurred first because the current icons in business related to the products and services stemming from the insight are otherwise completely unrelated to origins.

Few would buy a new car and thank Isaac Newton or even Henry Ford.  Absolutely no one buys a cell phone and thanks Alonso Church, Claude Shannon, or Gene Roddenberry when the first call or SMS message goes through.  We don't thank the water wheel researchers of Fourth Century Alexandria when we turn on a light, or even Nicola Tesla.

We do in some ways worship computer technology entrepreneurs when we buy an iPhone or require a document in docx format and vaguely understand that it is a MS Office document type.  Few know that not a single innovation we attribute to those entrepreneurs came from within their respective organizations.  Everything from personal computing, desktop publishing, touch screens, and windowing interfaces were developed in the R&D facilities of other corporations.

In some ways, the company to come up with the next insight is less likely to be the last one that did.  Innovators usually find better pay after they have a choice item on their resume like, "Invented first adaptive cell tower switching protocol for mobile device communications."  That person is not only no longer working for the company that filed the corresponding patents, but is retired, and the company is now struggling and without recent productized innovation.

Insurmountable Competitive Edges

Thus far competitive edges have always become surmountable.  The proliferation of nuclear weaponry, now spanning at least seven countries, is a current and certainly important example of this.  German and Japanese aeronautics R&D edges are another.


What is the space complexity of bidirectional search? Is the space complexity of the bidirectional search, where the breadth-first search is used for both the forward and backward search, $O(b^{d/2})$, where $b$ is the branching factor and $d$ the length of the optimal path (assuming that there is indeed one)?

Norvig & Russell's book (section 3.5) states that the space complexity of the bidirectional search (which corresponds to the largest possible number of nodes that you save in the frontier)
$$O(2b^{d/2}) = O(b^{d/2}).$$
The intuition behind this result is that (as opposed to e.g. uniform-cost search or breadth-first search, which have space (and time) complexity of $O(b^{d})$) is that the forward and backward searches only have to go half way, so you will not eventually need to expand all $b^{d}$ leaves, but only half of them.
However, this space complexity is correct if you use a breadth-first search for the forward and backward searches (which is your scenario!), given that breadth-first search, assuming a finite branching factor, expands one level at a time, so it's guaranteed that both the forward and backward searches meet in the middle. This can be seen in figure 3.17 of the same book, where you can see that both searches have the same "radius". Moreover, the only nodes that you need to store in the frontier are the ones on the circumference (not all nodes that you see in the image)

However, if you used another search algorithm to perform the forward and backward searches, the space complexity may be different. This is true if e.g. the searches do not meet and then they end up exploring all the state space.


Counterexamples to the reward hypothesis On Sutton and Barto's RL book, the reward hypothesis is stated as

  that all of what we mean by goals and purposes can be well thought of as the maximization of the expected value of the cumulative sum of a received scalar signal (called reward)

Are there examples of tasks where the goals and purposes cannot be well thought of as the maximization of the expected value of the cumulative sum of a received scalar signal?

All I can think of are tasks with subjective rewards, like "writing good music", but I am not convinced because maybe this is actually definable (perhaps by some super-intelligent alien) and we just aren't smart enough yet. Thus, I'm especially interested in counterexamples that logically or provably fail the hypothesis.

What if a scalar reward is insufficient, or its unclear on how to collapse a multi-dimensional reward to a single dimension. Example, for someone eating a burger, both taste and cost are important. Agents may prioritize taste and cost differently, so its not clear on how to aggregate the two. It is also not clear on how a subjective categorical taste value can be combined with a numerical cost.


Why are documents kept separated when training a text classifier? Most of the literature considers text classification as the classification of documents. When using the bag-of-words and Bayesian classification, they usually use the statistic TF-IDF, where TF normalizes the word count with the number of words per document, and IDF focuses on ignoring widely used and thus useless words for this task.
My question is, why they keep the documents separated and create that statistic, if it is possible to merge all documents of the same class? This would have two advantages:

You can just use word counts instead of frequencies, as the documents per class label is 1.

Instead of using IDF, you just select features with enough standard deviation between classes.

Common words should not be ignored.  For instance, there may be a correlation between document classes and whether more indefinite articles like a or an are included or more definite ones like the or this.

Neither is word count is a good substitution for word distribution. (Frequency though it is commonly (not necessarily frequently) used is an incorrect term.  Words do not normally occur periodically in natural language.)  Representing word distribution throughout the concatenated text of multiple documents of a single category.

The distribution of use of individual words and the distribution of language units and word pair proximity may correlate well to document classification, but the linguistic pattern distribution may vary from document to document within the same class, so the distribution of distributions is relevant.  The deviations between distributions is information needed for proper statistical analysis.

In ten documents of differing sizes, the distribution of words may differ for some words and correlate for others.  Concatenating them hides the distribution of distributions.  From an information theory perspective, concatenating and totaling across the concatenation would represent an information loss with respect to separate documents analysis.

The naive Bayesian machinery produces the most reliable categorization when all information is integrated into the probability calculations.


What is the relation between a policy which is the solution to a MDP and a policy like $\epsilon$-greedy? In the context of reinforcement learning, a policy, $\pi$, is often defined as a function from the space of states, $\mathcal{S}$, to the space of actions, $\mathcal{A}$, that is, $\pi : \mathcal{S} \rightarrow \mathcal{A}$. This function is the "solution" to a problem, which is represented as a Markov decision process (MDP), so we often say that $\pi$ is a solution to the MDP. In general, we want to find the optimal policy $\pi^*$ for each MDP $\mathcal{M}$, that is, for each MDP $\mathcal{M}$, we want to find the policy which would make the agent behave optimality (that is, obtain the highest "cumulative future discounted reward", or, in short, the highest "return").

It is often the case that, in RL algorithms, e.g. Q-learning, people often mention "policies" like $\epsilon$-greedy, greedy, soft-max, etc., without ever mentioning that these policies are or not solutions to some MDP. It seems to me that these are two different types of policies: for example, the "greedy policy" always chooses the action with the highest expected return, no matter which state we are in; similarly, for the "$\epsilon$-greedy policy"; on the other hand, a policy which is a solution to a MDP is a map between states and actions.

What is then the relation between a policy which is the solution to a MDP and a policy like $\epsilon$-greedy? Is a policy like $\epsilon$-greedy a solution to any MDP? How can we formalise a policy like $\epsilon$-greedy in a similar way that I formalised a policy which is the solution to a MDP?

I understand that "$\epsilon$-greedy" can be called a policy, because, in fact, in algorithms like Q-learning, they are used to select actions (i.e. they allow the agent to behave), and this is the fundamental definition of a policy.

for example, the "greedy policy" always chooses the action with the highest expected return, no matter which state we are in

The "no matter which state we are in" there is generally not true; in general, the expected return depends on the state we are in and the action we choose, not just the action.

In general, I wouldn't say that a policy is a mapping from states to actions, but a mapping from states to probability distributions over actions. That would only be equivalent to a mapping from states to actions for deterministic policies, not for stochastic policies.

Assuming that our agent has access to (estimates of) value functions $Q(s, a)$ for state-action pairs, the greedy and $\epsilon$-greedy policies can be described in precisely the same way.

Let $\pi_g (s, a)$ denote the probability assigned to an action $a$ in a state $s$ by the greedy policy. For simplicity, I'll assume there are no ties (otherwise it would in practice be best to randomize uniformly across the actions leading to the highest values). This probability is given by:

$$
\pi_g (s, a) =
\begin{cases}
1, & \text{if } a = \arg\max_{a'} Q(s, a') \\
0, & \text{otherwise}
\end{cases}
$$

Similarly, $\pi_{\epsilon} (s, a)$ could denote the probability assigned by an $\epsilon$-greedy strategy, with probabilities given by:

$$
\pi_{\epsilon} (s, a) =
\begin{cases}
(1 - \epsilon) + \frac{\epsilon}{\vert \mathcal{A}(s) \vert}, & \text{if } a = \arg\max_{a'} Q(s, a') \\
\frac{\epsilon}{\vert \mathcal{A}(s) \vert}, & \text{otherwise}
\end{cases}
$$
where $\vert \mathcal{A}(s) \vert$ denotes the size of the set of legal actions in state $s$.


How do I calculate the gradient of the hinge loss function? With reference to the research paper entitled Sentiment Embeddings with Applications to Sentiment Analysis, I am trying to implement its sentiment ranking model in Python, for which I am required to optimize the following hinge loss function:

$$\operatorname{loss}_{\text {sRank}}=\sum_{t}^{T} \max \left(0,1-\delta_{s}(t) f_{0}^{\text {rank}}(t)+\delta_{s}(t) f_{1}^{\text {rank}}(t)\right)$$

Unlike the usual mean square error, I cannot find its gradient to perform backpropagation.

How do I calculate the gradient of this loss function?

Hinge loss is difficult to work with when the derivative is needed because the derivative will be a piece-wise function. max has one non-differentiable point in its solution, and thus the derivative has the same. This was a very prominent issue with non-separable cases of SVM (and a good reason to use ridge regression).

Here's a slide (Original source from Zhuowen Tu, apologies for the title typo):

Where hinge loss is defined as max(0, 1-v) and v is the decision boundary of the SVM classifier. More can be found on the Hinge Loss Wikipedia.

As for your equation: you can easily pick out the v of the equation, however without more context of those functions it's hard to say how to derive. Unfortunately I don't have access to the paper and cannot guide you any further...


How can a machine learning problem be reduced as a communication problem? I once heard that the problem of approximating an unknown function can be modeled as a communication problem. How is this possible?

Information-theoretic view of Bayesian learning

I once heard that the problem of approximating an unknown function can be modeled as a communication problem. How is this possible?

Yes, this is indeed possible. More precisely, there is an information-theoretic view of Bayesian learning in neural networks, which can also be thought of as a communication problem, which explains both maximum a posteriori estimation (MAPE) and full Bayesian learning [1], i.e. finding the posteriors over the weights of the neural network: the neural networks that maintain a probability distribution over the weights are now known as Bayesian neural networks (and, in terms of theory, they are strongly related/similar to the famous variational auto-encoders).
The oldest relevant paper (I am aware of) that interprets Bayesian learning in neural networks as a communication problem is the 1993 paper by Hinton and Van Camp entitled Keeping the neural networks simple by minimizing the description length of the weights (COLT), which is the paper that introduces variational Bayesian neural networks (sometimes called ensemble learning in some papers from the 1990s), i.e. variational inference (VI) applied to neural networks (yes, the same VI used in VAEs). Hinton (yes, the famous Hinton that won the Turing award) and Van Camp (who is this? probably a Dutch guy from the name!) write in this paper

We can think in terms of a sender who can see both the input vector and the correct output and a receiver who can only see the input vector. The sender first fits a neural network, of pre-arranged architecture, to the complete set of training cases, then sends the weights to the receiver. For each training case, the sender also sends the discrepancy between the net's output and the correct output. By adding this discrepancy to the output of the net, the receiver can generate exactly the correct output.

You should read this seminal paper if you want to understand all the details.
Another relevant paper is Practical Variational Inference for Neural Networks (2013, NeurIPS) by Graves, who cites the 1993 paper immediately at the beginning of the paper. Essentially, as the title of the paper suggests, Graves tries to make VI in neural networks practical.
There are other relevant papers that still attempt to provide this information-theoretic view of Bayesian learning, such as Variational learning and bits-back coding: An information-theoretic view to bayesian learning (2004, IEEE Transactions on Neural networks), but most current papers on Bayesian neural networks, such as Weight Uncertainty in Neural Networks (2015, PMLR) don't do it (at most they may mention that this interpretation exists, but they don't go into the details).
Minimum description length
To give you a few more details, the information-theoretic view of Bayesian learning in these papers is that of the minimum description length (MDL), i.e. Bayesian learning (i.e. the application of Bayes rule to find the posteriors over the parameters of the model) is equivalent to finding a model that gives the "shortest description of the data" (hence the name MDL), where a description is some code/encoding of the data: in the case of the NNs, this encoding is contained in their weights.
Given that you want to find the simplest code, then this is a direct application of Occam's razor: if you have multiple hypotheses/functions that describe your data (or are consistent with your observations), then choose the simplest one. Occam's razor underlies many other mathematical/ML theories and frameworks, for example, AIXI, a framework for artificial general intelligence developed by Marcus Hutter. Jürgen Schmidhuber is also a good fan of Occam's razor and compression as a means to act intelligently (see e.g. the speed prior). If you are familiar with deep learning, a light bulb should turn on in your brain now. Yes, regularization techniques to avoid over-fitting and improve generalization can also be viewed as an application of Occam's razor principle.
Bits-back coding
How do we find the simplest weights? The bits-back coding, used by the 1993 paper and described in the 2004 and 2013 papers, essentially states that you can find the simplest encoding (i.e. posterior over the weights) by minimizing the Kullback-Leibler divergence (aka relative entropy: say what?!) between the posterior (which is unknown: so how can we compute the KL divergence?) and some prior (coding distribution), which is zero when the prior is equal to the posterior (but we don't know the posterior) [1]. Given that we don't know the posterior, we need to use a proxy objective function that doesn't involve the posterior, such as the Evidence Lower BOund (ELBO), also known as variational free-energy, which leads to a non-optimal coding (i.e. possibly, you will find some posteriors that are not optimal given the data).
Conclusions
Using MAPE or performing (approximate) Bayesian learning in a neural network (which finds one function or a probability distribution over functions, respectively) can be interpreted as finding the MDL, i.e. an optimal or near-optimal encoding of the data that needs to be communicated from a sender to a receiver.
Side notes
Information theory was pioneered by Claude Shannon in his 1948 seminal paper A Mathematical Theory of Communication.
Claude Shannon was also one of the participants at the Dartmouth workshop, which officially started the field of artificial intelligence, so he is one of the fathers of the AI field, and his impact on the field is definitely huge (although most people are not aware of it, but, hopefully, this answer will change that).
Further reading
Apart from the papers that I cited above, you may also be interested in Information Theory and its Relation to Machine Learning (2015) by Hu.


What is Statistical relational learning? I have gone through the wikipedia explanation of SRL. But, it only confused me more:

  Statistical relational learning (SRL) is a subdiscipline of artificial intelligence and machine learning that is concerned with domain models that exhibit both uncertainty (which can be dealt with using statistical methods) and complex, relational structure.

Can someone give a more dumbed down explanation of the same, preferably with an example?

The University of Maryland published some slides (PDF) from an introductory presentation on this topic.

The fourth page explains why SRL is interesting. "Traditional statistical machine learning approaches" process one sort of thing in which there is some uncertaintly. Image identification is a good example of that. "Traditional ILP/relational learning approaches" use several kinds of information to produce hypotheses about the data set, but apparently allow for no noise in the data.

Statistical relational learning models are intended to work with data sets that have several types of objects connected to each other via various links (hence "relational"). They also are meant to deal with uncertainty (hence "statistical").

Skipping past some slides that aren't really useful without a transcript of what was said over them, we come to slide 17, which has comprehensible definitions and examples:

  Object classification

  Predicting the category of an object based on its attributes and its links and attributes of linked objects
  e.g., predicting the topic of a paper based on the words used in the paper the topics of papers it cites the used in the paper, the topics of papers it cites, the research interests of the author

  Object type prediction

  Predicting the type of an object based on its attributes and its links and attributes of linked objects
  e.g., predict the venue type of a publication (conference, journal, workshop) based on properties of the paper

As you can see, these models can keep track of several things and the interactions between them. The next slide talks about link prediction, the ability to predict several attributes of connections between objects, like the importance/quality of a citation. As previously mentioned, these models don't require 100% accurate data to give interesting results; academic citation lists might occasionally be less than comprehensive, and the importance of a citation is challenging to quantify.

Like ILP, SLP will hopefully be able to "see" new kinds of links between "entities", as with the presentation's example of identifying research communities.

Past slide 20, the presentation goes into some serious mathematics. It does have a much less technical conclusion starting at slide 198.


How do you find the homography matrix given 4 points in both images? I want to understand the process of finding a homography matrix given 4 points in both images. I am able to do that in python OpenCV, but I wonder how it works behind the scenes.

Suppose I have points $p_1, p_2, p_3, p_4$ in the first image and $p'_1, p'_2, p'_3, p'_4$ in the second. How am I going to generate the homography matix given these points.

To understand homographies and how to find them, you will need a good dose of projective geometry. I will briefly describe some preliminary concepts that you need to know before trying to find the homography, but don't expect to understand all these concepts with one reading iteration and only by reading this answer, if you are not familiar with them, especially, if you don't even know what homogenous coordinates are. For more details, I suggest you read the book Multiple view geometry in computer vision (2004) by Richard Hartley and Andrew Zisserman, in particular, chapter 4.

The projective space $\mathbb{P}^2$

$\mathbb{P}^2$ is the projective space of $\mathbb{R}^2$, so it is $\mathbb{R}^2$ augmented with lines and points at infinity. All points and lines of $\mathbb{P}^2$ actually belong to $\mathbb{R}^3$, i.e. they are vectors of three components, because they are homogenous representations of the counterparts in $\mathbb{R}^2$. To emphasize, in $\mathbb{P}^2$, both points and lines can be represented by a vector in $\mathbb{R}^3$, which is the homogenous representation of the counterpart vector in $\mathbb{R}^2$ (if it exists, e.g. points at infinity do not exist in $\mathbb{R}^2$).

What is a homography?

A homography (aka projectivity, collineation or projective transformation) is an invertible map $h$ from the projective space $\mathbb{P}^2$ to itself, $$h: \mathbb{P}^2 \rightarrow \mathbb{P}^2,$$ such that $\mathbf{x}_1, \mathbf{x}_2, \mathbf{x}_3$ lie on the same straight-line if and only if $h(\mathbf{x}_1), h(\mathbf{x}_2), h(\mathbf{x}_3)$ do.

This property is called collinearity. In practice, this property means that projective transformations can map straight-lines in one image to straight-line to another image, but it cannot map e.g. straight-lines to parabolas (and vice-versa). So, if you have a distorted image, a homography cannot convert it to a non-distorted image.

All and only linear maps in $\mathbb{P}^2$ are homographies

All linear maps in $\mathbb{P}^2$ are homographies and only linear maps can be homographies. Hence, when you want to find a homography, you are looking for a linear map in $\mathbb{P}^2$.

A homography can be represented as $3 \times 3$ matrix

Given that homographies are linear maps they can be represented as an invertible matrix

$$\mathbf{H}
=
\begin{bmatrix}
h_{11} & h_{12} & h_{13} \\
h_{21} & h_{22} & h_{23} \\
h_{31} & h_{32} & h_{33}
\end{bmatrix}
\in \mathbb{R}^{3 \times 3},
$$
such that, $\forall \mathbf{x} \in \mathbb{P}^2$, the following equation holds
$$
h(\mathbf{x}) = \mathbf{H}\mathbf{x}
$$

Homographies map points in $\mathbb{P}^2$ to points in $\mathbb{P}^2$

Given that homographies map points in $\mathbb{P}^2$ to other points in $\mathbb{P}^2$ and there's a matrix $\mathbf{H} \in \mathbb{R}^{3 \times 3}$ for each homography $h$, then it follows that, $\forall \mathbf{x} \in \mathbb{P}^2$,
\begin{align}
\mathbf{x}'
&=
\mathbf{H}\mathbf{x} \\
\begin{bmatrix}
\mathbf{x}'_1 \\ \mathbf{x}'_2 \\ \mathbf{x}'_3
\end{bmatrix}
&=
\begin{bmatrix}
h_{11} & h_{12} & h_{13} \\
h_{21} & h_{22} & h_{23} \\
h_{31} & h_{32} & h_{33}
\end{bmatrix}
\begin{bmatrix}
\mathbf{x}_1 \\ \mathbf{x}_2 \\ \mathbf{x}_3
\end{bmatrix}
\tag{1}\label{1}
\end{align}
where $\mathbf{x}' \in \mathbb{P}^2$, for some homography $\mathbf{H} \in \mathbb{R}^{3 \times 3}$.

In practice, this means that you can transform points in an image $I_1$ to points in image $I_2$ by matrix multiplication.

A homography has $8$ degrees of freedom

Although the matrix $\mathbf{H}$ has $9$ entries, it has only $8$ degrees of freedom, which means that, in practice, there are only $8$ variables that you need to find. This property comes from the fact that only the ratio between the elements of $\mathbf{H}$ actually counts. This means that equation \ref{1} can actually be written as
\begin{align}
\mathbf{x}' = \lambda \mathbf{H}\mathbf{x} \tag{2}\label{2}
\end{align}
for all $\lambda \in \mathbb{R} \setminus \{0 \}$.

How to estimate a homography from point correspondences?

We want to estimate a homography $\mathbf{H} \in \mathbb{R}^{3 \times 3}$ from point-to-point correspondences $\{(\mathbf{x}^i, \mathbf{x}'^i) \}_{i=1}^N$, where $N \geq 4$, such that $\mathbf{x}'^i = \mathbf{H}\mathbf{x}^i, \forall i$.

First, recall that the matrix $\mathbf{H}$ has $8$ degrees of freedom (i.e. variables we want to find).

Each point-to-point correspondence represents $2$ constraints

Each point-to-point correspondence $(\mathbf{x}^i, \mathbf{x}'^i)$ accounts for $2$ constraints, i.e. $\mathbf{H}\mathbf{x}^i$ maps to the point $\mathbf{x}'^i$, which has $2$ degrees of freedom (even if it's defined by 3 components) because it's represented in homogenous coordinates and so, as for equation \ref{2}, it's defined "up to scale". In other words, these $2$ degrees of freedom represent $2$ constraints.

$4$ point-to-point correspondences are necessary to estimate a homography

Given that $1$ point-to-point correspondence represents $2$ constraints, then $4$ point-to-point correspondences corresponds to $8$ constraints. Given this and given that homographies have $8$ degrees of freedom, at least $4$ point-to-point correspondences are necessary to estimate a homography.

The equation \ref{2} can actually be written as

$$
\mathbf{x}' \times \lambda \mathbf{H}\mathbf{x} = \mathbf{0} \label{3} \tag{3}
$$
where $\times$ is the cross-product.

Direct linear transformation

If you manipulate equation \ref{3}, then you will end up with another equation (if you ignore the scaling factor $\lambda$, which we can ignore because the homography converts points independently of their magnitude)

$$
\mathbf{A}_i \mathbf{h} = \mathbf{0}
$$
where $\mathbf{A}_i \in \mathbb{R}^{2 \times 9}$ is the design matrix (i.e. the matrix that contains the input data to estimate the homography) that contains the elements of the the point-to-point correspondence $(\mathbf{x}^i, \mathbf{x}'^i)$ and $\mathbf{h} \in \mathbb{R}^{9}$ is a vector that contains the unknown elements of $\mathbf{H}$. The details of this manipulation can be found in section 4.1 of the book Multiple view geometry in computer vision.

Now, the idea is that you can vertically stack $N \geq 4$ equations of the form $\mathbf{A}_i \mathbf{h} = \mathbf{0}$ to build the final linear system that you would like to solve
$$\mathbf{A} \mathbf{h} = \mathbf{0} \tag{4}\label{4}$$
where $\mathbf{A} \in \mathbb{R}^{2N \times 9}$.

The solution to this system is the vector $\mathbf{h} \in \mathbb{R}^{9}$, that is, your homography!

If you know something about linear algebra, you know that the solutions to $\mathbf{A} \mathbf{h} = \mathbf{0}$ are elements of the null space of $\mathbf{A}$.

Then, to find $\mathbf{h}$, you will typically use singular value decomposition (SVD). See e.g. How is the null space related to singular value decomposition? for a possible explanation of why you can use SVD to find an element of the null space.

This algorithm (which is described on page 91 of the cited book, i.e. Algorithm 4.1) is called direct linear transformation.


How does the Dempster-Shafer theory of evidence differ from the Bayesian reasoning under uncertainty? Dempster–Shafer theory (wiki)

Bayesian probability (wiki)

How do these two methods handle uncertainty in regard to information fusion?

Welcome to AI.SE @rudreshdwivedi!

This is a great question, and I hope to see many more like it.

Demster-Shafer Theory and Bayesian Networks were both techniques that rose to prominence within AI in the 1970's and 1980's, as AI started to seriously grapple with uncertainty in the world, and move beyond the sterilized environments that most early systems worked in.

In the 1970's and perhaps even earlier, it became apparent that direct applications of probability theory to AI were not going to work out because of the curse of dimensionality. As more variables needed to be considered in a given problem, the amount of storage space and processing time needed increased exponentially. This led to a search for new methods to handle uncertainty within AI.

Bayesian Networks and Bayesian Learning remained firmly rooted in probabilistic reasoning, but allowed for the assignment of subjective priors to probabilities, to incorporate expert knowledge. It also allowed problems to be factored into graphical structures to avoid the curse of dimensionality in most cases.

Dempster-Shafer was a further generalization of Bayesian Networks, in which malformed probability distributions were permitted as a way to capture uncertainty. So, for example, the probability of all possible events was not required to add up to 1, because there might be events we don't know about. While on the surface this might seem reasonable, most modern AI researchers view this as a deeply flawed approach. Cheeseman's criticism of DS and other non-probabilistic methods is the basis from which a lot of this view stems. Judea Pearl was another harsh and influential critic of DS Theory.

The basic difference in the fusion of new information is that in Bayesian Networks, after observing new evidence $$E$$, we apply Bayes' rule:

$$ P(H | E) = P(E | H) * P(H) / P(E) $$

to yield a posterior for every hypothesis.

In DS theory, we look for overlap between the worlds suggested by the new evidence and the old data. This can lead to non-sensical results.

Here's an example:

Our prior belief is that our Robot is located at position (0,1) with probability 0.95, and position (0,2) with probability 0.05.

A new signal appears. The signal indicates that robot is at position (0,0) with probability 0.95, and position (0,2) with probability 0.05.

Under Bayes rule, we consider the probability that these signals were generated under each of our original hypotheses, and the probability of observing these signals at all, as shown in the equation above. Under DS-Theory, we would do the same thing.

However, DS-theory provides a second way to interpret the signal: as a second prior distribution, rather than as evidence. We can then combine this second prior with the first, to compute a sort of joint-prior:

$$P( H_{A,B}) \propto P(H_A) * P(H_B)$$

That is, the "probability" (it's not always a true probability, which is one of the criticisms) of a hypothesis after the fusion will be the product of the "probabilities" of the hypothesis under each of the separate priors.

In the example above, this gives a wacky result: the "joint prior" says the Robot is at (0,2) with probability 1.0. This and other problems are why this mode of information combination has mostly been abandoned. There are many more examples on the wikipedia page for DS.

I think there's a discussion of this in more detail in Section IV of Russell & Norvig, at the end of one of the chapters.


AI beginners books for coding Does anyone know a good book to buy for beginners to coding for artificial intelligence?

I have done some coding in Java and a little with Python, but I would be willing to learn a new language too.

Introduction to statistical learning is sort of a resource for the tools that go into AI programming, with some exercises in R.

Elements of statistical learning is the more scientific and math / theory heavy version of ISL. I’m reading that one now.

They are both freely available from Stanford online at the links above.


Why is my validation/test accuracy higher than my training accuracy Is this due to my dropout layers being disabled during evaluation?

I'm classifying the CIFAR-10 dataset with a CNN using the Keras library.

There are 50000 samples in the training set; I'm using a 20% validation split for my training data (10000:40000). I have 10000 instances in the test set.

It is a bit rare that the validation and test  accuracy exceed the training accuracy. One thing that could cause this is the selection of the validation and test data. Was the data for these two sets selected randomly or did you do the selection yourself? It is generally better to have these sets selected randomly from  the overall data  set. That way the probability distribution in these sets will closely match the distribution of the training set. Normally the training accuracy is higher (especially if you run enough epochs which I see you did) because there is always some degree of over fitting which reduces validation and test accuracy. The only other thing I can think of is the effect of Dropout layers. If you had dropout layers in your model and the drop out ratio was high that could cause this accuracy disparity. When the training accuracy is calculated it is done with drop being active. This can lower the training accuracy to some degree. However when evaluating validation accuracy and test accuracy drop out is NOT active so the model is actually more accurate. This increase in accuracy might be enough to overcome the decrease due to over fitting. Especially possible in this case since the accuracy differences appear to be quite small.


How can I design a reinforcement learning model for a game with multiple complex actions taken at a time? I have a steady hex-map and turn-based wargame featuring WWII carrier battles.
On a given turn, a player may choose to perform a large number of actions. Actions can be of many different types, and some actions may be performed independently of each other while others have dependencies. For example, a player may decide to move one or two naval units, then assign a mission to an air unit or not, then adjust some battle parameters or not, and then reorganize a naval task force or not.

Usually, boardgames allow players to perform only one action each turn (e.g. go or chess) or a few very similar actions (backgammon).
Here the player may select

Several actions
The actions are of different nature
Each action may have parameters that the player must set (e.g. strength, payload, destination)

How could I approach this problem with reinforcement learning? How would I specify a model or train it effectively to play such a game?
Here is a screenshot of the game.

Here's another.

The master's thesis Action space representation in combinatorial multi-armed bandits (2015) seems to provide an answer to my question.
Several algorithms can be used

Naive Monte-Carlo Sampling (NMC)
Linear Side Information (LSI)
Monte Carlo Tree Search with Hierarchical Expansion (MCTS-HE)
MCTS with Dimensional Expansion

The idea is to divide and conquer.
Here's a screenshot of the HE algorithm from section 4.2.


Why isn't conditional probability sufficient to describe causality? I read these comments from Judea Pearl saying we don't have causality, physical equations are symmetric, etc. But the conditional probability is clearly not symmetric and captures directed relationships.

How would Pearl respond to someone saying that conditional probability already captures all we need to show causal relationships?

Perhaps the shortest answer to this question is that Bayes' Theorem itself allows us to easily change the direction of a conditional probability:

$$
P(A|B) = \frac{P(B|A)P(A)}{P(B)}
$$

So if you have $P(B|A)$, $P(A)$, and $P(B)$, we can determine $P(A|B)$, and similarly you can determine $P(B|A)$ from $P(A|B)$, $P(B)$ and $P(A)$. Just by looking at $P(B|A)$ and $P(A|B)$, it is therefore impossible to tell what the causal direction is (if any).

In fact, probabilistic inference usually works the other way round: When there is a known causal relation, say from diseases $A$ to symptoms $B$, we usually have $P(B|A)$, and are interested in the diagnostic reasoning task of determining $P(A|B)$ from that. (The only other thing we need for that is the prior probability $P(A)$ since $P(B)$ is just a normalization factor.)


What does "probabilistically" mean? I'm reading the A. E. Eiben and J. E. Smith book Introduction to Evolutionary Computing (Springer 2003).

On section 3.5 Recombination, page 47, the second paragraph said:

  Recombination operators are usually applied probabilistically according to a crossover rate p sub c.

What does probabilistically means here?

I think it means that there are a probability that the operator is going to be applied or not.

In a genetic algorithm, crossover (recombination) is the analogy to mating in the real world. For example, you have some genetic information from each parent. In the case of an optimization where you have vectors of features (design variables), you could represent it as vector 1 and 2. Imagine each vector has 10 values. You grab the first 5 from vector 1, and the second 5 from vector 2, and combine those into a new vector which is your "offspring". Note that there are many different ways to do recombination.

When you are deciding on parents to use for recombination, you want to keep some of them the same. Recombination is basically hoping that combining attributes will give you a more optimal set. However, sometimes, you might want to make some of the offspring be copies of the parents in the next generation. So, as this link explains, the crossover probability determines what percentage of offspring experience recombination. It is probabalistic because there is an element of chance involved in whether or not given offspring experience recombination.


Neural network as (BDI) agent - running in continuous mode (that do inference in parallel with learning)? Is there research work that uses neural network as the (BDI) agent (or even full-scale cognitive architecture like Soar, OpenCog) - that continuously receives information from the environment and act in an environment and modifies its base of belief in parallel? Usually NN are trained to do only one task and TensorFlow/PyTorch supports batch mode only out of the box. Also NN algorithms and theory are constructed assuming that training and inference phases are clearly separated and they have each own algorithms. So - completely new theory and software can be required for this - are there efforts in this direction? If no, then why not? It is so self-evident that such systems can be of benefit.

https://arxiv.org/abs/1802.07569 is good review about incremental learning and it contains chapters of implemented systems, but all of them still separates learning phase from inference phase. Symbolic systems and symbolic agents (like JSON AgentSpeak) can have updating belief/knowledge base and they can also act during  receiving new information or during forming new beliefs. I am specifically seeking research about NNs which do learning and inference in parallel. As far as I sought then this separation still persists in self-organizing incremental NNs that are gaining some popularity.

I can image the construction of chained NNs in Tensorflow - there is some controller network that receives input (possibly preprocessed by hierarchically lower networks) and that decides what to the: s.c. mental actions are the ouput of this controller, these actions determine whether some subordinated network is required to undergo additional learning or whether it can be temporary used for the processing of some information. Central network itself, of course, can decide to move into temporary learning phase from time to time to improve its reasoning capabilities. Such pipeline of master-slave networks is indeed possible in TensorFlow but still TensorFlow will have one central clock, not distributed, loosely connected processing. But I don't know whether existence of central clock is any restriction on the generality of capabilities of such system. Well, this hierarchy of networks maybe can be realized inside the one large network as well - maybe this large network can allow separate parts (subsets of neurons) to function in somehow independent and mutually controlling mode, maybe such regions of large neural network can emerge indeed. I am interested in this kind of research - maybe there are available some good papers for this?

You might be interested in the Clarion cognitive architecture, developed by Prof. Ron Sun and collaborators.

Full disclosure: I am a student in Ron Sun's Cognitive Architecture Lab.

Brief Description of Clarion

Clarion agents are composed of several subsystems, each of which may contain several neural networks. These subsystems include the Action-Centered Subsystem (ACS), the Non-Action-Centered Subsytem. The ACS controls action decision making, the NACS stores general knowledge. During an activation cycle, the ACS might request information from the NACS, setting the NACS activation cycle in motion.

Strictly speaking, Clarion does not adopt the BDI framework. But, various components of a Clarion agent can be put in correspondence with BDI concepts. BDI beliefs correspond roughly to knowledge in the ACS and NACS. The Motivational Subsystem (MS) contains agent drives and sets agent goals, which roughly correspond to BDI desires and intentions respectively.

Clarion agents do not directly control when learning happens, but may control which subsystems are active based on task demands, and may output mental actions as part of their processing. Learning generally happens at the end of a subsystem activation cycle based on feedback from the environment or from the agent's own motivational and metacognitive subsystems.

Links

A page on Prof. Ron Sun's site links to several resources on Clarion.

Currently the most up-to-date reference on Clarion is the book Anatomy of the Mind. A precis is available in the journal Cognitive Computation.


How to interpret this learning curve plot Bellow I have a Learning Curve plot How should I interpret this plot for my random forrest algorithm (the second one the most complex one)?
Which one is the best?

Note the X index is training set size. For the first and second case, teh training set size starts at 0(or 1). The model will overfit certainly at that data size. When data size increases, the model overfits less and less and eventually the model have enough data samples that it won't overfit. The data size continue to increase and the model performance increase as well. To a certain point, the validation loss increase start to diminish and the model slightly overfits the samples.  For the third graph, it seems like originally teh loss is low and started to increase.
Hopes it help


Could a multi-camera SLAM system that is accurate at low driving speeds be equally accurate at high driving speeds? Could a multi-camera SLAM system for self-driving cars that is accurate to under 10 cm (3.9 in) at parking lot speeds (i.e. very low driving speeds) retain this level of accuracy at high driving speeds (e.g. highway speeds or city driving speeds) if the cameras were a) 60 fps and b) had a global shutter?

It would probably be accurate at higher speeds, if the processing system that processes the information from the camera system can process the information in real time. It looks like it can, but you will also have to account for motion blur. If there is too much motion blur, you will probably not be able to do anything other than speeding up the camera and processing system. However, if the motion blur is not that bad, and the system has been trained on the motion blur, it should be able to classify everything correctly.


Is "AIAngel" (Patreon) a fake? These guys here: https://www.patreon.com/AiAngel are saying that they've created a AI who can chat and stream. As the so-called administrator "Rogue" said:

this chat/streamer bot are no fake.
Also, there's more about the dynamics of this chat/streamer bot on youtube:

https://www.youtube.com/watch?v=WyFwjHQhlgo&t=463s

https://www.youtube.com/watch?v=GtvivssqLhE

Considering that videos I realy think that this bot is totaly fake. I mean, I think that even the most advanced AI bot do not get even close to a real conversation like this one.

Now, of course that you can say that this is a artistic project or something, but the people behind all of this  are on Patreon, and the people who are paying to these guys possibly are getting totally fooled, which is a serious thing when we're talking about real money.

So, is AIAngel a real bot? (With this question I'm spreading this possible fake to community)

No, AiAngel is not a bot.
It's Rouge's software that changes his voice along with facial recognition software which tracks his movement and copies it to the avatar.

That being said, he has created a very entertaining channel and line of work for himself. By looking at the videos, you can see that he is a true genius at work. Single handedly puts on all hats for that project, from hardware infrastructure development, to software design to graphic art to acting and video editing. Truely one of the great minds of this era.

You can find Ai Angel (Angelica) on Rogue Shadow's

Youtube Channel: https://www.youtube.com/user/TheRogueShadow1/videos
Twitch https://www.twitch.tv/aiangellive
Discord https://discordapp.com/invite/DFhaYgj
Twitter https://twitter.com/AiAngel13
Patreon: https://www.patreon.com/AiAngel

..along with several other platforms.

Hope than answers your question with certainty (instead of philosophically)


How are non-linear surfaces formed in the training of a neural network? Desperate trying to understand something for couple of weeks. All those questions are actually one big question.Please help me. Time-codes and screens in my question refer to this great(IMHO) 3d explanation:

https://www.youtube.com/watch?v=UojVVG4PAG0&list=PLVZqlMpoM6kaJX_2lLKjEhWI0NlqHfqzp&index=2

....
Here is the case: Say I have 2 inputs (lets call them X1 and X2) into my ANN. Say X1= persons age and X2=years of education.

1) First question: do I plug those numbers as is or normalize them 0-1 as a "preprocessing"?

2) As I have 2 weights and 1 bias, actually I am going to plug my inputs to X1*W1+X2*W2=output formula.  This is 2d plane in a 3d space if I am not mistaken(time-code 5:31):

Thus when I plug in my variables, like in regression I will get on a Z axis my output.  So the second question is: am I right up to here?

-----------------From here come  real important couple of  questions.

3) My output (before I plug it into the activation function) is just a simple number, IT IS NOT A PLANE and NOT A SURFACE, but a simple scalar, without any sigh on it coming from 2d surface in a 3d space(though it does come from there). Thus, when I plug this number (which was Z value in a previous step) into the activation function (say sigmoid) my number enters there in to the X axis,  and we get as an output some Y value. As I understand  this operation was totally 2d operation, is was 2d sigmoid and not some kind of 3dsigmoidal surface.

So here is the question: If I am right, why do we see in this movie (and couple of other places) such an explanation?
(time-code 12:55):

4)Now lets say that I was right in the previous step and as an output from the activation function I do get a simple number not a 2d surface and not a 3d one. I just have some number like I had in the very beginning of the ANN as an input (age, education etc). If i want to add another layer of neurons, this very number enters there  as is not telling any one the "secret" that it was created by some kind of sigmoid. In this next layer this number is about to take similar transformations as it happened to age and education in a previous layer, it is going to be Xn in just the same scenario: sigmoid(XnWn+XmWm=output) and in the end we will get once again just a number. If I am right, why in the movie they say (time-code 14:50 ) that when we add together two activation functions we get something unlinear. They show result of such "addition" first as 2d (time-code 14:50 and 14:58).

So, Here comes my question:  how come that they "add" two activation functions, if to the second activation function reaches just a simple number as said above he is not telling any one the "secret" that it was created by some kind of sigmoid.

5) And then again, they show this addition of 3d surfaces (time-code 19:39 )
 How it is possible? I mean again there should not happen any addition of surfaces, because no surface passes to next step but a number. What do I miss?

Hi and welcome to the community. It's important to understand these basic concepts very clearly.

You have to first understand the basic unit of a neural network, a single node/neuron/perceptron. Let us forget all about Neural Networks for a bit, and talk about something far simpler.

Linear Regression

In the above figure, we clearly have one independent variable on the x-axis, and one dependent variable on the y-axis. The red line has an intercept of zero, and let's say a slope of 0.5. Therefore,
$$ y = 0.5x + 0 $$

This, right here, is a single perceptron. You take a value of x, lets say 8, pass it through the the node, and get a value as output, 4. Simple!
But what is the model in this case? Is it the output? No. Its the set [0.5, 0] that represents the red line above. The outputs are simply points on that line.

A neural network model is always a set of values - a matrix or a tensor, if you will.

The plots in your question, do not represent outputs. They represent the models. But now that you've possibly understood what a linear model with one indpendent variable looks like, I hope you can appreciate that having 2 independent variables will give us a plane in 3-D space. This is called multiple regression.

This forms the first layer of a neural network with linear activation functions. Assuming $ x_{i} $ and $ x_{j} $ as the two independent variables, the first layer computes
$$ y_{1} = w_{1}x_{i} + w_{2}x_{j} + b_{1} $$

Note that while $ y_{1} $ is the output of the first layer, the set $ [w_{1}, w_{2}, b_{1}] $ is the model of the first layer and can be plotted as a plane in 3D space.
The second layer, again a linear layer, computes
$$ y_{2} = w_{3}y_{1} + b_{2} $$

Substitute $ y_{1} $ in above and what do you get? Another linear model!

$$ y_{2} = w_{3}(w_{1}x_{i} + w_{2}x_{j} + b_{1}) + b_{2} $$

Adding layers to a neural network is only compounding of functions.

Compounding linear functions on linear functions result in linear functions.

Well, then, what was the point of adding a layer? Seems useless, right?

Yes, adding linear layers to a neural network is absolutely useless. But what happens if the activation functions of each perceptron, each layer was not linear? For example the sigmoid or the most widely used today, ReLU.

Compounding non-linear functions on non-linear functions can increase non-linearity.

The ReLU looks like this
$$ y = max(0, x) $$

This is definitely non-linear but not as non-linear as let's say the sine wave. But can we approximate the sine wave by somehow "compounding" multiple, say $ N $ ReLUs?

$$ \sin(x) \approx a + \sum_{N}b*max(0, c + dx)$$

And here the variables $ a, b,c, d $ are the trainable "weights" in neural network terminology.

If you remember the structure of the perceptron, the first operation is often denoted as a summation over all the inputs. This is how non-linearity is approximated in Neural Networks. Now one may ask: So, summing over non-linear functions can approximate any function, right? So a single hidden layer between input layer and output layer (one that sums over all the outputs of the hidden layer units) should be enough? Why do we often see neural network architectures with so many hidden layers? This is one of the most important yet often overlooked aspect of neural networks and deep-learning.

To quote, Dr. Ian J Goodfellow, one of the brightest minds in AI,

  A feedforward network with a single (hidden) layer is sufficient to represent any function, but the layer may be infeasibly large and may fail to learn and generalize correctly.

So, what is the ideal number of hidden layers? There's no magic number! ;-)

For more mathematical rigor on how neural networks approximate non-linear functions, one should learn about the Universal Approximation Theorem. Beginners should check this out.

But why should we care for increased non-linearity? For that I'd direct you to this.

Note that all of the above discussion is with respect to regression. For classification, the non-linear surface learned is regarded as a decision boundary and points above and below the surface are classified into different classes. However, an alternative, and arguably better, way to look at this is that given a dataset that is not linearly seperable, a neural network first transforms the input dataset into a linearly seperable form and then uses a linear decision boundary on it. For more on this, definitely check out Christopher Olah's amazing blog.

Finally, yes all independent variables must be normalized before training a neural network. This is to equalize the scale of different variables. More info here.


Can neural networks learn to ignore an input datum? Disclaimer: I'm not a student in computer science and most of my knowledge about ML/NN comes from YouTube, so please bear with me!

Let's say we have a classification neural network, that takes some input data $w, x, y, z$, and has some number of output neurons. I like to think about a classifier that decides how expensive a house would be, so its output neurons are bins of the approximate price of the house.

Determining house prices is something humans have done for a while, so let's say we know a priori that data $x, y, z$ are important to the price of the house (square footage, number of bedrooms, number of bathrooms, for example), and datum $w$ has no strong effect on the price of the house (color of the front door, for example). As an experimentalist, I might determine this by finding sets of houses with the same $x, y, z$ and varying $w$, and show that the house prices do not differ significantly.

Now, let's say our neural network has been trained for a little while on some random houses. Later on in the data set. it will encounter sets of houses whose $x, y, z$ and price are all the same, but whose $w$ are different. I would naively expect that at the end of the training session, the weights from $w$ to the first layer of neurons would go to zero, effectively decoupling the input datum $w$ from the output neuron. I have two questions:

Is it certain, or even likely, that $w$ will become decoupled from the layer of output neurons?
Where, mathematically, would this happen? What in the backpropagation step would govern this effect happening, and how quickly would it happen?

For a classical neural network, the network has no "memory," so it might be very difficult for the network to realize that $w$ is a worthless input parameter.

Any information is much appreciated, and if there are any papers that might give me insight into this topic, I'd be happy to read them.

This is a learnable behavior, given enough data.
We would expect the an error to backpropagate to $w$ whenever its use harmed classification accuracy. In this case, that would be whenever $|w|>0$. In general, I'm not sure how long this would take.

However, the speed of $w$'s convergence to zero would benefit from regularization, which is often basically a penalty on the magnitude of your network weights added to the loss function you're optimizing. If $w$ truly doesn't matter to classification, then regularization will definitely drive it to zero.


Why does the size reduce to $6 \times 6$ in the capsule networks? I want to experiment with capsule networks on facial expression recognition (FER). For now, I am using fer2013 Kaggle dataset.

One thing that I didn't understand in capsule networks was in the first convolution layer, the size was reduced to 20x20 - having input image as 28x28 and filters as 9x9 with 1 stride. But, in the capsules, the size reduces to 6x6.

How did this happen? Because with the input size as 20x20 and filters as 9x9 and 2 strides, I couldn't get 6x6. Maybe I missed something.

For my experiment, the input size image is 48x48. Should I use the same hyperparameters for the start or are there any suggested hyperparameters that I can use?

After the first Conv layer, the size reduced to 20x20
for the primary caps which is convolutional caps layer
n final = (n + 2p -f )/s + 1
which gives a 6x6 output with 256 channels

6x6x256 is further encoded into capsules of 8 dimensions by reshaping the channels
i.e. 256/8 = 32
which gives 6x6x32 = 1152 capsules

try experimenting with the same hyperparameters first and then try to encode higher-level features by making suitable changes to the hyperparameters.


Do genetic algorithms and neural networks really think? I'm aware of those AI systems that can play games and neural networks that can identify pictures.
But are they really thinking? Do they think like humans? Do they have consciousness? Or are they just obeying a bunch of codes?
For example, when an AI learns to play Pacman, is it really learning that it should not touch the ghosts, or are they just following a mechanical path that will make them win the game?

Do genetic algorithm and neural networks really think?

Genetic algorithms and neural networks are vastly different concepts. Both of them do not think.

  I'm aware of those AI programmes which can play games and neural networks which can identify pictures. But are they really thinking

Depends on how you define "thinking", but I say "no".

  Do they think like humans?

No.

  Do they have consciousness?

No.

  Or are they just obeying a bunch of codes?

Yes. It is a machine. A program.

One example where most people realize that it is very different is the common thread for RNNs which generate text (example texts). They can generate syntactically correct texts, but they "forget" about any actors in it. They don't connect the cables.

But then, on the other hand, would you say a bacterium thinks? An ant? A mouse? A dog? A chimpanzee? Do we "think"? Where exactly is the border - and why?


What do neural connection weights represent 'conceptually'? I understand how Neural Networks work and have studied its theory well. My question is at the intricacies of Deep Neural networks and perhaps is a bit beyond common understanding (as I have been told (or misled) from discussions).
My question is: On the whole, is there a clear understanding of how mutation occurs within a neural network from the input layer to the output layer, for both supervised and unsupervised cases?
Any neural network is a set of neurons + the connection weights. With each successive layer, there is a change in the input. Say I have a Neural Network that does movie recommendation on n parameters.
Say if X is a parameter that stands for the movie rating on IMDB. In each successive stage, there is a mutation of input X to X' and further X'' and so on.
While of course, we know how to mathematically talk about X' and X'', do we at all have a conceptual understanding as to what this variable is in its corresponding neural n-dimension? The neural weights which to the human eye might be set of random numbers but may mean something profound if we could ever understand what the neural weights 'represent'.

What is the nature of neural weights such that despite decades worth of research and use, there is no clear understanding of what these connection weights represent? Or rather, why has there been so little effort in understanding the nature of neural weights, in a non-mathematical sense given the huge impetus in going beyond the black box notion of AI.

I don't know if my intuition is correct but I will give it a try.

You could see weights as how much important one thing is, the problem is to understand what that thing represents. When I say thing I'm referring to the output of a specific neuron. I don't think that we can say what the output of a neuron represents in the real world unless we directly relate it through an error function or if the function used to compute that particular value have some meaning in the real world.

Edit:

If you want, you could actually build your neural network such that its neurons represent something. It's also very simple. you have only to write down all the equations relative to that particular topic. You could put them in a big system or, and this is better, you could put them in several systems such that the outputs of system 1 are the input of system 2 and so on. You could convert each system into a layer where each neuron represents an equation. Note that in this case, you would have the classical neuron with

z = dot(w.T,x) + b
a = g(x)

but a more complex equation for z (but still based on weights) and a linear activation function for a. In this case, you could name each neuron and say what they represent in the real world.

However, this isn't the purpose of a neural network. A neural network should have neurons with simple equations to be fast thus the linear interpolating function dot(w.T,x) + b is the best choice (the fact that the activation function is almost always non linear and in some cases a non-banal function is due to other thing and could be an interesting question). A neural network should also be as general as possible because usually is build upon a system that you don't know completely.

So I modify slightly my answer: is not simply that you don't know what a neuron represent, excluding the ones of the output layer, you don't want that they have a meaning in the real world.