How to make a distinction between item feature and environment feature? My data is stock data with features such as stocks' closing prices.I am curious to know if I can put the economy feature such as 'national interest rate' or 'unemployment rate' besides each stocks' features.
Data:
  Date  Ticker  Open  High  Low  Close  Interest  Unemp.
  1/1    AMZN    75    78     73   76     0.015     0.03
  1/2    AMZN    76    77     72   72     0.016     0.03
  1/3    AMZN    72    78     76   77     0.013     0.03
  ...    ...     ...   ...    ...  ...    ...       ...
  1/1    AAPL    104   105    102  102    0.015     0.03
  1/2    AAPL    102   107    104  105    0.016     0.03
  1/3    AAPL    105   115    110  111    0.013     0.03
  ...    ...     ...   ...    ...  ...    ...      ...

As you can see from the table above, daily prices of AMZN and AAPL are different but the Interest and Unemployment rates are the same. Can I feed the data to my neural network like the table above?
In other words, can I put the individual stocks' information besides the environment feature such as interest rates?

I am curious to know if I can put the economy feature such as 'national interest rate' or 'unemployment rate' besides each stocks' features.

The variables are macro-econometric and they, in general, seem to have some influence on stocks' prices. This inclusion might as well increase your model's prediction accuracy. You can definitely use them as predictors. As mentioned in comments - Experimentation is the way to go.

  Can I feed the data to my neural network like the table above?

In general, you can have any kind of numeric variables as input to a neural network. Things will work out fine. The important thing is the selection of relevant predictor variables that, potentially, have some relationship with the response variable.


Is there an open-source implementation for graph convolution networks for weighted graphs? Currently, I'm using a Python library, StellarGraph, to implement GCN. And I now have a situation where I have graphs with weighted edges. Unfortunately, StellarGraph doesn't support those graphs

I'm looking for an open-source implementation for graph convolution networks for weighted graphs. I've searched a lot, but mostly they assumed unweighted graphs. Is there an open-source implementation for GCNs for weighted graphs?

A Comprehensive Survey on Graph Neural Networks (2019) presents a list of ConvGNN's. All of the following accept weighted graphs, and three accept those with edge weights as well:

And below is a series of open source implementations of many of the above:


Which explainable artificial intelligence techniques are there? Explainable artificial intelligence (XAI) is concerned with the development of techniques that can enhance the interpretability, accountability, and transparency of artificial intelligence and, in particular, machine learning algorithms and models, especially black-box ones, such as artificial neural networks, so that these can also be adopted in areas, like healthcare, where the interpretability and understanding of the results (e.g. classifications) are required.
Which XAI techniques are there?

Explainable AI and model interpretability are hyper-active and hyper-hot areas of current research (think of holy grail, or something), which have been brought forward lately not least due to the (often tremendous) success of deep learning models in  various tasks, plus the necessity of algorithmic fairness & accountability.
Here are some state of the art algorithms and approaches, together with implementations and frameworks.

Model-agnostic approaches

LIME: Local Interpretable Model-agnostic Explanations (paper, code, blog post, R port)

SHAP: A Unified Approach to Interpreting Model Predictions (paper, Python package, R package). GPU implementation for tree models by NVIDIA using RAPIDS - GPUTreeShap (paper, code, blog post)

Anchors: High-Precision Model-Agnostic Explanations (paper, authors' Python code, Java implementation)

Diverse Counterfactual Explanations (DiCE) by Microsoft (paper, code, blog post)

Black Box Auditing and Certifying and Removing Disparate Impact (authors' Python code)

FairML: Auditing Black-Box Predictive Models, by Cloudera Fast Forward Labs (blog post, paper, code)

SHAP seems to enjoy high popularity among practitioners; the method has firm theoretical foundations on co-operational game theory (Shapley values), and it has in a great degree integrated the LIME approach under a common framework. Although model-agnostic, specific & efficient implementations are available for neural networks (DeepExplainer) and tree ensembles (TreeExplainer, paper).

Neural network approaches (mostly, but not exclusively, for computer vision models)

The Layer-wise Relevance Propagation (LRP) toolbox for neural networks (2015 paper @ PLoS ONE, 2016 paper @ JMLR, project page, code, TF Slim wrapper)

Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization (paper, authors' Torch code, Tensorflow code, PyTorch code, yet another Pytorch implementation, Keras example notebook, Coursera Guided Project)

Axiom-based Grad-CAM (XGrad-CAM): Towards Accurate Visualization and Explanation of CNNs, a refinement of the existing Grad-CAM method (paper, code)

SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability (paper, code, Google blog post)

TCAV: Testing with Concept Activation Vectors (ICML 2018 paper, Tensorflow code)

Integrated Gradients (paper, code, Tensorflow tutorial, independent implementations)

Network Dissection: Quantifying Interpretability of Deep Visual Representations, by MIT CSAIL (project page, Caffe code, PyTorch port)

GAN Dissection: Visualizing and Understanding Generative Adversarial Networks, by MIT CSAIL (project page, with links to paper & code)

Explain to Fix: A Framework to Interpret and Correct DNN Object Detector Predictions (paper, code)

Transparecy-by-Design (TbD) networks (paper, code, demo)

Distilling a Neural Network Into a Soft Decision Tree, a 2017 paper by Geoff Hinton, with various independent PyTorch implementations

Understanding Deep Networks via Extremal Perturbations and Smooth Masks (paper), implemented in TorchRay (see below)

Understanding the Role of Individual Units in a Deep Neural Network (preprint, 2020 paper @ PNAS, code, project page)

GNNExplainer: Generating Explanations for Graph Neural Networks (paper, code)

Benchmarking Deep Learning Interpretability in Time Series Predictions (paper @ NeurIPS 2020, code utilizing Captum)

Concept Whitening for Interpretable Image Recognition (paper, preprint, code)

Libraries & frameworks
As interpretability moves toward the mainstream, there are already frameworks and toolboxes that incorporate more than one of the algorithms and techniques mentioned and linked above; here is a partial list:

The ELI5 Python library (code, documentation)

DALEX - moDel Agnostic Language for Exploration and eXplanation (homepage, code, JMLR paper), part of the DrWhy.AI project

The What-If tool by Google, a feature of the open-source TensorBoard web application, which let users analyze an ML model without writing code (project page, code, blog post)

The Language Interpretability Tool (LIT) by Google, a visual, interactive model-understanding tool for NLP models (project page, code, blog post)

Lucid, a collection of infrastructure and tools for research in neural network interpretability by Google (code; papers: Feature Visualization, The Building Blocks of Interpretability)

TorchRay by Facebook, a PyTorch package implementing several visualization methods for deep CNNs

iNNvestigate Neural Networks (code, JMLR paper)

tf-explain - interpretability methods as Tensorflow 2.0 callbacks (code, docs, blog post)

InterpretML by Microsoft (homepage, code still in alpha, paper)

Captum by Facebook AI - model interpetability for Pytorch (homepage, code, intro blog post)

Skater, by Oracle (code, docs)

Alibi, by SeldonIO (code, docs)

AI Explainability 360, commenced by IBM and moved to the Linux Foundation (homepage, code, docs, IBM Bluemix, blog post)

Ecco: explaining transformer-based NLP models using interactive visualizations (homepage, code, article).

Recipes for Machine Learning Interpretability in H2O Driverless AI (repo)

Reviews & general papers

A Survey of Methods for Explaining Black Box Models (2018, ACM Computing Surveys)

Definitions, methods, and applications in interpretable machine learning (2019, PNAS)

Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead (2019, Nature Machine Intelligence, preprint)

Machine Learning Interpretability: A Survey on Methods and Metrics (2019, Electronics)

Principles and Practice of Explainable Machine Learning (2020, preprint)

Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challenges (keynote at 2020 ECML XKDD workshop by Christoph Molnar, video & slides)

Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI (2020, Information Fusion)

Counterfactual Explanations for Machine Learning: A Review (2020, preprint, critique by Judea Pearl)

Interpretability 2020, an applied research report by Cloudera Fast Forward, updated regularly

Interpreting Predictions of NLP Models (EMNLP 2020 tutorial)

Explainable NLP Datasets (site, preprint, highlights)

Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges

eBooks (available online)

Interpretable Machine Learning, by Christoph Molnar, with R code available

Explanatory Model Analysis, by DALEX creators Przemyslaw Biecek and Tomasz Burzykowski, with both R & Python code snippets

An Introduction to Machine Learning Interpretability (2nd ed. 2019), by H2O

Online courses & tutorials

Machine Learning Explainability, Kaggle tutorial

Explainable AI: Scene Classification and GradCam Visualization, Coursera guided project

Explainable Machine Learning with LIME and H2O in R, Coursera guided project

Interpretability and Explainability in Machine Learning, Harvard COMPSCI 282BR

Other resources

explained.ai blog

A Twitter thread, linking to several interpretation tools available for R

A whole bunch of resources in the Awesome Machine Learning Interpetability repo


Having trouble understanding some of the details of R-CNN (first one) Here is what I understand (what I think I understand).

We first train out model on our images using transfer learning.

So now we have a pre-trained model.

For each image in out dataset, we compute selective search on it, which makes 2000 region proposals.
    These 2000 region proposals are feed through our pre-trained NN ,

However we only collect the output (feature maps) from the last convolution layer. These outputs are saved to a hard-disk.

These feature maps are fed into a SVM for another round of training, were another label, "no object" is added.

We also have regression model that trains based on the window coordinates that we also annotated.

So we have SVN and a regression model (two models) that we train.

1)Is the above correct?

2) Are each of these 2000 region proposals hand-labeled (correct label (cat, dog etc) or no-object) before feeding it into the SVM?

3) Is the regression model tied into the SVM model? Basically out loss is a combination of both regression coords and SVM classification?

Yes, your description is mostly aligned with the original paper. However, Figure 1 shows they use features from the second fully connected layer (not the last convolutional layer).
Yes. The training is performed by exploiting hand-annotated bounding boxes. If a proposal has IoU less than 0.3, then it is considered as a negative example (cf. section 2.3).
No. Appendix C explains that box regression parameters are learned separately from the SVM model.


How to build a commercial image captioning system? Image Captioning is a hot research topic in the AI community. There are considerable image captioning models for research usage such as NIC, Neural Talk 2 etc. But can these research models be used for commercial purpose? Or we should build much more complex structured ones for commercial usage? Or if we can make some improvements based these models to meet the business applications situation? If so, what improvements should we take? Are there any existing commercial Image Captioning applications can be referenced?

Thinking Commercially

A commercial solution will need to be able to ascertain, continuously verify, and utilize the best of options for captioning learning models.

Each fairly successful image captioning learning models could be placed in an adapter to provide a common training, optimizing, testing, evaluation, and usage interface.  The hot swappable container-addon mega pattern used in peripheral device installation, RAID, J2EE containers, browsers, and other containment sub-systems can be applied.

System Description

The system acceptance criteria being as follows.

A new model could be added without stopping or starting the system if tauted as successful by others
Any model can be deleted without starting or stopping the system if it has not been performing well
Each possible state of each model added can be given some percentage of the total system resources
Various training processes and hyper parameter tuning processes can be applied to any of those model-state combinations
A-B testing can be done on any model-state combination
Stats can be displayed on any model-state combination any model overall or any state across all added models
The interface for each model requires that its associated model produces an array of caption-reliability pairs with zero ore more pairs
The reliability measure that is included in the pair with the caption is an indication of the model's assessment of appropriateness to the image
The most appropriate caption is picked from the proposals of the various models

States can be idle or any of these.

Training
Testing
Evaluation
Hyper parameter tuning
In use

For example, for the two learning models usable in captioning systems suggested in this question, NIC and Neural Talk 2 we could have a system resource allocation like this:

30% NIC Training
5% NIC Hyper parameter tuning
45% NT2 In use
15% NT2 Evaluation
5% NT2 Hyper parameter tuning

Samples may be pulled from a pool of samples that have been vetted.  That pool may be augmented by real images passing through the system, filtered in accordance with security criteria to avoid external attempts at control.

In the assignment of resources, the sample pool selection criteria must be specified.  If the system is already at 100%, the model-state combinations from which the resources shall be drawn must also be specified.

Handling Multiple Output Options

Since there may be more than one model in use and each model may have zero, one, or multiple caption suggestions for each image, each with a reliability measure, the outputs must be analyzed to provide the best choice to associate with the image being analyzed.  Additional system criteria must cover this process scenario.  For any given image, final evaluation must follow the following general guidelines.

If multiple models produce similar or exactly the same captions, they most be weighted higher in final evaluation.
If the model's reliability is proven in actual use based on feedback from end users, the model's output must be weighed higher accordingly
Re-entrant models (such as reinforcement learning network models) must have access to the end user feedback for additional learning even in in-use state
Clear winners are chosen
Close races are disambiguated through trained functionality
Exact ties are broken through pseudo random index generation

Another artificial network may be placed at the output and appropriate encoding and normalization may be applied before training so that a properly trained network, converged using a quantification of the above additional criteria, can select the best caption from the options for each image.

Phased Development Approach

Phase one of such a system would likely require manual handling of model-state allocation.  Phase two would be semi-automation.  The location of new models would still require expert attention.  Perhaps further in the future a hunt for new models could be automated too.


What are the best classifiers for this type of data? I would like to classify a dataset Credit Scoring, which is composed of 21 attributes, some of them are numeric and others are boolean.

For the output, I want to know if they have a good or bad credit based on those attributes, without calculating any numeric value for the credit score.

I am using Weka for this task. However, I am not sure what are the best/ideal classifiers for that kind of datasets.

Anyone here can put me in the right direction?

Well, it depends on the structure of the data. The best way is to try all the intelligent models like naive bayes, random forest, svm with different parameters by grid search. There is no model works best all the time for classification. However,  neural network (named Multilayer Perceptron on weka) is supposed to be better if it is set correctly.


Relationship between input range and channel means, standard deviations for CNNs So, I'm using a pretrained pnasnet5large model to do some image classification (https://github.com/Cadene/pretrained-models.pytorch/blob/master/pretrainedmodels/models/pnasnet.py)

In the file, it says that the input range is in [0,1] (i'm assuming pixel values of input images). The images I have are already in this range.
The channel means and standard deviation for RGB channels are stated as [0.5, 0.5, 0.5], [0.5, 0.5, 0.5] respectively.
Now when I use the torchvision.transforms.Normalize(https://pytorch.org/docs/stable/torchvision/transforms.html#torchvision.transforms.Normalize) to normalize the images using the stated means and standard deviations, the pixel values get to the range [-1,1].

The code I wrote for normalization:

transforms.Normalize([0.5, 0.5, 0.5],[0.5, 0.5, 0.5])

I believe I'm missing something fundamental. Should I normalize the images or should I not? Thanks!

Standardization of the pixel values would bring it mean close to 0 and standard deviation as 1.
Normalization will squeeze the values between 0 and 1.
For RGB images, divide each channel i.e each pixel value by 255.
For reference regarding Standardization see here and for normalization see here.


How large should the replay buffer be? I'm learning DDPG algorithm by following the following link: Open AI Spinning Up document on DDPG, where it is written

  In order for the algorithm to have stable behavior, the replay buffer should be large enough to contain a wide range of experiences, but it may not always be good to keep everything.

What does this mean? Is it related to the tuning of the parameter of the batch size in the algorithm?

In order for the algorithm to have stable behavior, the replay buffer should be large enough to contain a wide range of experiences, but it may not always be good to keep everything.

The larger the experience replay, the less likely you will sample correlated elements, hence the more stable the training of the NN will be. However, a large experience replay also requires a lot of memory and it might slow training. So, there is a trade-off between training stability (of the NN) and memory requirements.

The authors of the linked article state (right after the sentence above)

  If you only use the very-most recent data, you will overfit to that and things will break; if you use too much experience, you may slow down your learning. This may take some tuning to get right.


What is the main contribution of the paper Disentangling by Factorising? Considering the paper Disentangling by Factorising, in addition to introducing a new model for Disentangled Representation Learning, FactorVAE (see figure), what is the main theoretical contribution provided by the paper?

One of the core contributions presented in the paper consists of understanding at a deeper level the objective function used in Beta VAE and improving it

More specifically, the authors started from Beta VAE OF

$$\frac{1}{N} \sum_{i=1}^{N}\left[\mathbb{E}_{q\left(z | x^{(i)}\right)}\left[\log p\left(x^{(i)} | z\right)\right]-\beta K L\left(q\left(z | x^{(i)}\right) \| p(z)\right)\right]$$

which consists of the classical reconstruction loss and the regularization term which steers the Latent Code PDF towards a target PDF.

Developing the second term

$$\mathbb{E}_{p_{\text{data}}(x)}[K L(q(z | x) \| p(z))]=I(x ; z)+K L(q(z) \| p(z))$$

they observed there are 2 terms which push in different directions:

$I(x,z)$ is the mutual information between the input data and the code, and we do not want to penalize this as in fact we want this to be as high as possible, so that the decode contains as much information as possible about the input
$KL(q(z) || p(z))$ is the KL divergence term which if penalized pushes the latent code PDF $q(z)$ towards the prior $p(z)$ which we choose as factorized
So penalizing the second term in Beta VAE OF means also penalizing the mutual information hence ultimately loosing reconstruction capability

The authors then propose a new OF which fixes this

$$\frac{1}{N} \sum_{i=1}^{N}\left[\mathbb{E}_{q\left(z | x^{(i)}\right)}\left[\log p\left(x^{(i)} | z\right)\right]-K L\left(q\left(z | x^{(i)}\right)|| p(z)\right)\right] -\gamma K L(q(z) \| \bar{q}(z))$$

In the rest of the paper they run experiments and show they have improved the disentanglement of the representation (according to the new metric they propose in the paper) while preserving reconstruction hence improving the reconstruction vs disentanglement tradeoff with respect to the Beta VAE paper


In what RL algorithm category is MiniMax? Q-learning is a temporal-difference method and Monte Carlo tree search is a Monte Carlo method. In what category is MiniMax?

I think you are looking at it from the wrong direction, min-max is just a planning algorithm, decision strategy, in the sense that you are describing other algorithms/methods it does not have a category. For example, you have negamax algorithm which is in a sense the same thing the Monte Carlo Search Tree is to Monte Carlo. Min-max category is game theory really.

Now you should be thinking about RL algorithms in another way, and this is taxonomy:

So if you think about methods, you mentioned, let's put them in the right place:

TD methods in general - model free
Monte Carlo methods - model free
MinMax - model-based (that could be discussed but it definitely needs access to a world model)


Can we use ML to do anything else other than predicting (in the case of mathematical problems)? (The math problem here just serves as an example, my question is on this type of problems in general).
Given two Schur polynomials, $s_\mu$, $s_\nu$, we know that we can decompose their product into a linear combination of other Schur polynomials.
$$s_\mu s_\nu = \sum_\lambda c_{\mu,\nu}^\lambda s_\lambda$$
and we call $c_{\mu,\nu}^\lambda$ the LR coefficient (always an non-negative integer).
Hence, a natural supervised learning problem is to predict whether the LR coefficient is of a certain value or not given the tuple $<\mu, \nu, \lambda>$. This is not difficult.
My question is: can we either use ML/RL to do anything else other than predicting (in this situation) or extract anything from the prediction result? In other words, a statement like "oh, I am 98% confident that this LR coefficient is 0" does not imply anything mathematically interesting?

There are quite a few examples of papers where they try and 'teach' neural networks to 'learn' how to solve math problems. Most of the time, sadly, it comes down to training on a large dataset after which the network can 'solve' the sort of basic problems, but is unable to generalize this to larger problems. That is, if you train a neural network to solve addition, it will be inherently constrained by the dataset. It might be able to semi-sufficiently solve addition with 3 or even 4 digits, depending on how big your dataset is, but throw in an addition question containing 2 10 digit numbers, and it will almost always fail.
The latest example that I can remember where they tried this is in the General Language Model GPT-3, which was not made to solve equations per se, but does 'a decent job' on the stuff that was in the dataset. Facebook AI made an 'advanced math solver' with a specific architecture that i have not looked into which might disproof my point, but you can look into that.
In the end, this comes down to 'what is learning' and 'what do you want to accomplish'. Most agree that these network are not able to generalize beyond their datasets. Some might say that not being able to generalize does not mean that it is not learning. It might just be learning slower. I believe that these models are inherently limited to what is presented in the dataset. Given a good dataset, it might be able to generalize to cases 'near and in-between', but I have yet to see a case where this sort of stuff generalizes to cases 'far outside' the dataset.


Why is the logarithm of the standard deviation used in this implementation of proximal policy optimization? I am currently writing my bachelor thesis, which is an implementation of proximal policy optimization. Sometimes, I hit a wall because of the gaps in my mathematical knowledge. However, implementing the algorithm helped me to understand the math behind the algorithm.
Unfortunately, I still have a question.
When the action space is continuous, I am using the normal distribution (same as in the PPO implementation by Spinning up). In the mentioned implementation, the logarithm of the standard deviation is used initially to give the same probability to all of the possible action, then they use the standard deviation when choosing an action. Why do we use the logarithm? Why not directly use simply the standard deviation?
I know that the logarithm is easier when it comes to the computations, but I can not see the benefits of the logarithm in the Spinning up implementation.
class MLPGaussianActor(Actor):

    def __init__(self, obs_dim, act_dim, hidden_sizes, activation):
        super().__init__()
        log_std = -0.5 * np.ones(act_dim, dtype=np.float32)
        self.log_std = torch.nn.Parameter(torch.as_tensor(log_std))
        self.mu_net = mlp([obs_dim] + list(hidden_sizes) + [act_dim], activation)

    def _distribution(self, obs):
        mu = self.mu_net(obs)
        std = torch.exp(self.log_std)
        return Normal(mu, std)

    def _log_prob_from_distribution(self, pi, act):
        return pi.log_prob(act).sum(axis=-1)    # Last axis sum needed for Torch Normal distribution

As you have mentioned using log is nicer because it makes multiplications to additions etc etc(it helps in numerical stability issues). But I think over here, the reason they are doing it like that is because of enforcing a simple constraint in a much more simpler way. In the __init__ we are noticing that the log_std is being formulated instead of the std itself. Would it be wrong if you formulated the std itself? No. But it would be a bit messy to have some constrained imposed on it. For example, if we modeled std it could have been possible for the std to become 0 or negative. Well that's not a correct values of a std. But over here that is being enforced automatically, this kind of little things makes the learning for the model easier. Even if the log_std becomes negative or zero it does not matter because the exp will take care of it by exponentiating it to a positive number.


What is the purpose of Decoder mask (triangular mask) in Transformer? I'm trying to implement transformer model using this tutorial.  In the decoder block of the Transformer model, a mask is passed to "pad and mask future tokens in the input received by the decoder". This mask is added to attention weights.
import tensorflow as tf

def create_look_ahead_mask(size):
    mask = 1 - tf.linalg.band_part(tf.ones((size, size)), -1, 0)
    return mask

Now my question is, how is doing this step (adding mask to the attention weights) equivalent to revealing the words to model one by one? I simply can't grasp the intuition of it's role. Most tutorials won't even mention this step like it's very obvious. Please help me understand. Thanks.

The Transformer model presented in this tutorial is an auto-regressive Transformer. Which means that prediction of next token only depends on it's previous tokens.
So in order to predict next token, you have to make sure that only previous token are attended. (If not, this would be a cheating because model already knows whats next).
So attention mask would be like this
[0, 1, 1, 1, 1]
[0, 0, 1, 1, 1]
[0, 0, 0, 1, 1]
[0, 0, 0, 0, 1]
[0, 0, 0, 0, 0]
For example: If you are translating English to Spanish
Input: How are you ?
Target: < start > Como estas ? < end >
Then decoder will predict something like this
< start > (it will be given to decoder as initial token)
< start > Como
< start > Como estas
< start > Como estas ?
< start > Como estas ? < end >
Now compare this step by step prediction sequences to attention mask given above, It would make sense now to you


How do you explain Hebbian Learning in an intuitive way? http://www.cs.bham.ac.uk/~jxb/INC/l5.pdf

  The neuropsychologist Donald Hebb postulated in 1949 how biological
  neurons learn:

  “When an axon of cell A is near enough to excite a cell B and
  repeatedly or persistently takes part in firing it, some growth
  process or metabolic change takes place on one or both cells such that
  A’s efficiency as one of the cells firing B, is increased.”

  In more familiar terminology, that can be stated as the Hebbian
  Learning rule:

  If two neurons on either side of a synapse (connection) are activated
  simultaneously (i.e. synchronously), then the strength of that synapse
  is selectively increased.

  Mathematically, we can describe Hebbian learning as:

  Here, η is a learning rate coefficient, and x are the outputs of the
  ith and jth elements.

Now, my question is, what do all these descriptions mean?

Is Hebbian Learning applicable for single-neuron networks?
What does it mean by "two neurons on either side of a synapse"?
Why/when would two neurons activate simultaneously?
What does they mean by elements?

Hebbian learning is trying to answer "How the strength of the synapse between 2 neurons evolve over period of time based on the activity of the 2 neurons involved.". You can call it learning if you think learning is just strengthening of synapses.
The connection between 2 neurons are called synapse. A synapse is the point where the axons of a neuron meets with the dendrites of another neuron.
The 2 neurons are connected to each other but they are also connected to other neurons as well. So it may happen that both the neurons gets activated by their other connected neurons at the same time.
The elements refer the 2 neurons. The equation is basically saying that the synapse strength between i and j neuron at time (n+1) depends on its strength at time n plus activations of the i and j neurons at time n.


Is k-fold cross-validation more effective than splitting the dataset into training and test datasets to prevent overfitting? I want to prevent my model from overfitting. I think that k-fold cross-validation (because it is doing this each time with different datasets) may be more effective than splitting the dataset into training and test datasets to prevent overfitting, but a colleague (who has little experience in ML) says that, to prevent overfitting, the 70/30% split performs better than the cross-validation. In my opinion, k-fold cross-validation provides a reliable method to test the model performance.

Is k-fold cross-validation more effective than splitting the dataset into training and test datasets to prevent overfitting?
I am not concerned with computational resources.

K-fold cross-validation is probably preferred in terms of completeness and generalization: you ensure that the system has seen the complete dataset for training. However, in deep learning this is often not feasible due to time and power constraints. They can both be used, and there is not one better than the other. It really depends on the specific case, the size of the dataset and the time and hardware available. Note that overfitting can be (partially) remedied by things such as dropout.

To be fair: it is fine to have a discussion about this with your colleagues, but as so often there is no one correct answer. If you really want proof, you can test it out and compare them. But performance-wise (i.e. the model's predictive power), the difference will be small.


Can artificial intelligence be thought of as optimization? In this video an expert says, "One way of thinking about what intelligence is [specifically with regard to artificial intelligence], is as an optimization process."

Can intelligence always be thought of as an optimization process, and can artificial intelligence always be modeled as an optimization problem? What about pattern recognition? Or is he mischaracterizing?

A good answer to this question depends on what you want to use the labels for.

When I think about "optimization," I think about a solution space and a cost function; that is, there are many possible answers that could be returned and we can know what the cost is of any particular answer.

In this view, the answer is "yes"--pattern recognition is a case where each pattern is a possible answer, and the optimization method is trying to find the one where the cost is lowest (that is, where the answer matches what you want it to match).

But most interesting optimization problems are characterized by exponential solution spaces and clean cost functions, and so can be thought of more as 'search' problems, whereas most pattern recognition problems are characterized by simple solution spaces and complicated cost functions, and it might feel unnatural to put the two of them together.

(In general, I do think that optimization and intelligence are deeply linked enough that optimization power is a good measure of intelligence, and certainly a better measure of the practical use of intelligence than pattern recognition.)


How can I handle overfitting in reinforcement learning problems? So this is my current result (loss and score per episode) of my RL model in a simple two players game:

I use DQN with CNN as a policy and target networks. I train my model using Adam optimizer and calculate the loss using Smooth L1 Loss.

In a normal "Supervised Learning" situation, I can deduce that my model is overfitting. And I can imagine some methods to tackle this problem (e.g. Dropout layer, Regularization, Smaller Learning Rate, Early Stopping). But is that solution will also work in RL problem? Or are there any better solutions to handle overfitting in RL?

Overfitting refers to a model being stuck in a local minimum while trying to minimise a loss function. In Reinforcement Learning the aim is to learn an optimal policy by maximising (not always but almost) a reward function rather than minimising a loss function, so you can't really talk of overfitting as in the supervised scenario, but you can definitely talk about sub-optimal policies.
If we think of a specific task like avoiding stationary objects, a simple sub-optimal policy would be to just stay still without moving at all, or moving in circles if the reward function was designed to penalise lack of movements.
The way to avoid an agent to learn sub-optimal policies is to find a good compromise between exploitation, i.e. the constant selection of the next action to take based on the maximum expected reward possible, and exploration, i.e. a random selection of the next action to take regardless of the rewards. Here's a link to an introduction to the topic:
Exploration and Exploitation in Reinforcement Learning
It is worth mentioning that sometimes an agent can actually outsmart humans though, some examples are reported in this paper The Surprising Creativity of Digital Evolution. I particularly like the story of the insect agent trained to learn to walk while minimising the contact with the floor surface. The agent surprisingly managed to learn to walk without touching the ground at all. When the authors checked what was going on they discovered that the insect leaned to flip itself and then walk using its fake 'elbows' (fig7 in the linked paper). I add this story just to point out that most of the time the design of the reward function is itself even more important than exploration and exploitation tuning.


Handling emotion in informal text (Hi vs HIIIIII!!!!)? This is a question related to Neural network to detect "spam"?.
I'm wondering how it would be possible to handle the emotion conveyed in text. In informal writing, especially among a juvenile audience, it's usual to find emotion expressed as repetition of characters. For example, "Hi" doesn't mean the same as "Hiiiiiiiiiiiiiii" but "hiiiiii", "hiiiiiiiii", and "hiiiiiiiiii" do.

A naive solution would be to preprocess the input and remove the repeating characters after a certain threshold, say, 4. This would probably reduce most long "hiiiii" to 4 "hiiii", giving a separate meaning (weight in a context?) to "hi" vs "long hi".

The naivete of this solution appears when there are combinations. For example,
haha vs hahahahaha or lol vs lololololol. Again, we could write a regex to reduce lolol[ol]+ to lolol. But then we run into the issue of hahahaahhaaha where a typo broke the sequence.

There is also the whole issue of Emoji. Emoji may seem daunting at first since they are special characters. But once understood, emoji may actually become helpful in this situation. For example,  may mean a very different thing than , but  may mean the same as  and .

The trick with emojis, to me, is that they might actually be easier to parse. Simply add spaces between  to convert  to     in the text analysis. I would guess that repetition would play a role in training, but unlike "hi", and "hiiii", Word2Vec won't try to categorize  and  as different words (as I've now forced to be separate words, relying in frequency to detect the emotion of the phrase).

Even more, this would help the detection of "playful" language such as , where the  emoji might imply there is anger, but alongside  and especially when repeating  multiple times, it would be easier for a neural network to understand that the person isn't really angry.

Does any of this make sense or I'm going in the wrong direction?

These kinds of repetitions in text can place recurrence demands on learning algorithms that may or may not be handled without special encoding.

Hi.
Hiiii!
HIIIIIIIII
Hi!!!!!!!!!!!!!!

These have the same meaning on one level, but different emotional content and therefore different correlations to categories when detecting the value of an email, which in the simplest case is the placement of a message in one of two categories.

Pass to a recipient
Archive only

This is colloquially called spam detection, although not all useless emails are spam and some messages sent by organizations that broadcast spam may be useful, so technically the term spam is not particularly useful. The determinant should usually be the return on investment to the recipient or the organization receiving and categorizing the message.

  Is reading the message and potentially responding likely of greater value than the cost of reading it?

That is a high level paraphrase of what the value or cost function must represent when AI components are employed to learn about or track close to (in continuous learning) some business or personal optimality.

The question proposes a normalization scheme that truncates long repetitions of short patterns in characters, but truncation is necessarily destructive. Compression of some type that will both preserve nuance and work with the author's use of Word2Vec is a more flexible and comprehensive approach.

In the case of playful sequences of characters it is anthropomorphic to imagine that an artificial network will understand playfulness or anger, however existing learning devices can certainly learn to use character sequences that humans would call playful or angry in the function that emerges to categorize the message containing them. Just remember that model free learning is not at all like cognition, so the term understanding is placing an expectation on the mental capacities of the AI component that the AI component may not possess.

Since no indication that a recurrent or recursive network will be used but rather the entire message is represented in a fixed width vector, so the question becomes which of these two approaches will produce the best outcomes after learning.

Leaving the text uncompressed so that an 'H' character followed by ten 'i' characters is distinct as a word from an 'H' character followed by five 'i' characters
Compressing the text to "Hi [9xi]" and "Hi [4xi]" respectively or some such word bifurcation.

This second approach produces reasonable behavior with other cases mentioned, such as "" pre-processed into " [2x]". What the algorithm in Word2Vec will do with each of these two choices and how its handling of them will affect outcomes is difficult to predict. Experiments must be run. Three things are advisable courses of action.

Build a test fixture to allow quick evaluation of outcomes for various trials.
Experiment with dilligence. Don't leave any potentially interesting case untried.
Label as much production data as reasonably possible and use that as well as the canned data so that the above options can be evaluated in permuted combinations with the differences in pattern distribution between the canned and live data.


How is a deep neural network different from other neural networks? How is a neural network having the "deep" adjective actually distinguished from other similar networks?

The difference is mostly in the number of layers.

For a long time, it was believed that "1-2 hidden layers are enough for most tasks" and it was impractical to use more than that, because training neural networks can be very computationally demanding.

Nowadays, computers are capable of much more, so people have started to use networks with more layers and found that they work very well for some tasks.

The word "deep" is there simply to distinguish these networks from the traditional, "more shallow" ones.


Which paper introduced the term "softmax"? Nowadays, the softmax function is widely used in deep learning and, specifically, classification with neural networks. However, the origins of this term and function are almost never mentioned anywhere. So, which paper introduced this term?

The paper that appears to have introduced the term "softmax" is Training Stochastic Model Recognition Algorithms as Networks can Lead to Maximum Mutual Information Estimation of Parameters (1989, NIPS) by John S. Bridle.
As a side note, the softmax function (with base $b = e^{-\beta}$)
$$\sigma (\mathbf {z} )_{i}={\frac {e^{-\beta z_{i}}}{\sum _{j=1}^{K}e^{-\beta z_{j}}}}{\text{ for }}i=1,\dotsc ,K {\text{ and }}\mathbf {z} =(z_{1},\dotsc ,z_{K})\in \mathbb {R} ^{K}$$
is very similar to the Boltzmann (or Gibbs) distribution
$$
p_i=\frac{e^{- {\varepsilon}_i / k T}}{\sum_{j=1}^{M}{e^{- {\varepsilon}_j / k T}}}
$$
which was formulated by Ludwig Boltzmann in 1868, so the idea and formulation of the softmax function is quite old.


What exactly are the "parameters" in GPT-3's 175 billion parameters and how are they chosen/generated? When I studied neural networks, parameters were learning rate, batch size etc. But even GPT3's ArXiv paper does not mention anything about what exactly the parameters are, but gives a small hint that they might just be sentences.

Even tutorial sites like this one start talking about the usual parameters, but also say "model_name: This indicates which model we are using. In our case, we are using the GPT-2 model with 345 million parameters or weights". So are the 175 billion "parameters" just neural weights? Why then are they called parameters? GPT3's paper shows that there are only 96 layers, so I'm assuming it's not a very deep network, but extremely fat. Or does it mean that each "parameter" is just a representation of the encoders or decoders?

An excerpt from this website shows tokens:

In this case, there are two additional parameters that can be passed
to gpt2.generate(): truncate and include_prefix. For example, if each
short text begins with a <|startoftext|> token and ends with a
<|endoftext|>, then setting prefix='<|startoftext|>',
truncate=<|endoftext|>', and include_prefix=False, and length is
sufficient, then gpt-2-simple will automatically extract the shortform
texts, even when generating in batches.

So are the parameters various kinds of tokens that are manually created by humans who try to fine-tune the models? Still, 175 billion such fine-tuning parameters is too high for humans to create, so I assume the "parameters" are auto-generated somehow.
The attention-based paper mentions the query-key-value weight matrices as the "parameters". Even if it is these weights, I'd just like to know what kind of a process generates these parameters, who chooses the parameters and specifies the relevance of words? If it's created automatically, how is it done?

Parameters is a synonym for weights, which is the term most people use for a neural networks parameters (and indeed in my experience it is a term that machine learners will use in general whereas parameters is more often found in statistics literature). Batch size, learning rate etc. are hyper-parameters which basically means they are user specified, whereas weights are what the learning algorithm will learn through training.


Can we define the AI singularity mathematically? The "AI Singularity" or "Technological Singularity" is a vague term that roughly seems to refer to the idea of:

Humans can design algorithms
Humans can improve algorithms
Eventually algorithms we design might end up being as good as humans at designing and improving algorithms
This might lead to these algorithms designing better versions of themselves, eventually becoming far more intelligent than humans. This improvement would continue to grow at an increasing rate until we reach a "singularity" where an AI is capable of making technological progress at a rate far faster than we could ever imagine

Also known as an Intelligence Explosion. This rough idea has been heavily debated as to its feasibility, how fast it'll take (if it does happen), etc.

However I'm not aware of any formal definitions of the concept of "singularity". Are there any? If not, do we have close approximations?

I have seen AIXI and the Gödel machine, but these both require some "reward signal" — it is unclear to me what reward signal one should choose to bring about a singularity, or really how those models are even relevant here. Because even if we had an oracle that can solve any formal problem given to it, it's unclear to me how we could use that to cause a singularity to happen (see this question for more discussion on that note).

I found someone that has done this thing! You can hear a good explanation in Marcus Hutter's answer to this question about rewards given to AIXI. He describes a work that seems to be referring to this paper:

Universal Knowledge-Seeking Agents for Stochastic Environments

I'll edit this answer later with a full explanation of the approach, but essentially the idea is that you use an AIXI model that does optimal reinforcement learning, giving it a reward that is based on information gained (phrased in a careful way to avoid a few common pitfalls). As a result, it learns to choose actions that give it the most information possible to predict the impacts of it's actions. This results in a "scientist" like behaviour, and you could imagine it doing things like turing the entire earth into a supercollider to better understand some physics laws if it decides that is the best approach for gaining maximum information. It would probably also do plenty of very unethical psycology experiements, for example, if it ended up deciding that human actions were important to predict and understand.

It's not a "safe" singularity in that sense, but that's okay, I didn't require that. It's at least a formal definition. It requires doing some uncomputable things, but the hope of future research is that we can make close enough approximations to those uncomputable things to be good enough anyway.

I feel this theory is lacking any explanation of how feasable such a task is since it uses uncomputable agents, so I won't accept it yet, but it's the best answer I've seen so far. And I'll be watching future research closely to see if they can get a better handle on feasability, there seems to be quite a bit of work that has gone on in finding computable approximations to AIXI. The reason I care about feasability is because it is very relevant for mathmatically deciding how plausable something like an "intelligence explosion" actually is. So if a theory doesn't talk about feasability, it is missing out on a big piece of this question. Still this theory seems hopeful. For example, maybe there are fundamental computational limits to maximizing some reward functions, and we can prove that even certain levels of approximations for this reward function aren't computable. That would be a really interesting negative result.

In general, I think the idea of using reinforcement learning, then choosing a reward function that tries to capture something instrinsic (such as "curiosity") is a very good approach to trying to formally define the singularity. I look forward to seeing other potential reward functions defined in the future, I don't expect this to be the only one.


Does a bias also have a chance to be dropped out in Dropout layer? Suppose that you have 80 neurons in a layer, where one neuron is bias. Then  you add a dropout layer after the activation function of this layer.

In this case, does it have a chance to drop out the bias neuron, or does the dropout only affect the other 79 weight neurons?

Only the non-bias ones,

It is discouraged to include the bias weights under norm penalty regularization for example, so why should it be included in the drop-out regularization scheme?

Drop out can be implemented by multiplying units with zero and the bias term is rather special. The bias term determines the distance from the origin the linear decision boundary of the node implements. It is included in every sum, but it does not receive any inputs.

So to take your example, say you have 80 nodes per layer in an MLP, where one is a bias node, the output of these layers will consist of 79 nodes every time. This output is then put under the drop-out effect.

From Goodfellows et al. Deep Learning book under the regularization chapter, present online: "http://www.deeplearningbook.org/contents/regularization.html", they write that they implement the drop-out by canceling some outputs to zero:

"
In most modern neural networks, ... , we can effectively remove a unit from a network by multiplying its output value by zero. ... . Here, we present the dropout algorithm in terms of multiplication by zero for simplicity, but it can be trivially modiﬁed to work with other operations that remove a unit from the network.
"

Drop-out by cancelling outputs, the output from the activation function, has nothing to do with the bias vector. It is not receiving any inputs.
Therefore I think the implementation typically only deals with output-to-input connections, and the bias vector does not receive inputs so it can safely be let alone from the drop-out process.


Bellman optimality equation in semi Markov decision process I wrote a Python program for a simple inventory control problem where decision epochs are equally divided (every morning) and there is no lead time for orders (the time between submitting an order until receiving the order). I use the Bellman equations, and solve them by policy iteration (Dynamic Programming).
Now I want to consider lead time (between 1 to 3 days with equal probabilities). As I understand, the problem is defined by Semi Markov Decision Process for considering sojourn time in each state. I am confused about the Bellman equations in this scenario because we don't know exactly when the order will be received and is it necessary to discount the reward for day two or three?

The core problem here is state representation, not estimating return due to delayed response to actions on the original state representation (which is no longer complete for the new problem). If you fix that, then you can solve your problem as a normal MDP, and base calculations on single timesteps. This allows you to continue using dynamic programming to solve it, provided the state space remains small enough.
What needs to change is the state representation and state transitions. Instead of orders resulting in immediate change of stock levels, they become pending changes, and for each item you will have state representation for the amount of current stock, plus amount of stock in each lead time category. State transitions will modify expected lead time for each amount of pending stock as well as amount of current stock.
Your lead time categories will depend on whether the agent knows the lead time immediately after making an order:

If lead times are known, track remaining time until items arrive 1,2 or 3 days. These categories will be assigned by the enviroment following the order, then lead time will transition down on each day deterministically. A 1 day lead time will transition to in stock, 2 day lead will transition to 1 day etc.

If lead times are not known, but probabilities of them are, track time since the order was made. This will be 0, 1 or 2 days. Although you don't know when an order will arrive, you know the probabilities for state transition - e.g. items in 0 days have a 1 in 3 chance of transitioning to "in stock" and a 2 in 3 chance of transitioning to 1 days.

This makes the state space larger, but is less complex than moving to the Semi MDP representation. For instance, doing it this way means that you can still work with single time step transitions and apply dynamic programming in a standard way.
In general, if the environment has a delayed response to actions, then the  best way to maintain Markov trait is to add relevant history of actions taken to the state. The added state variables can either be a direct list of the relevant actions, or something that tracks the logical consequence of those actions.


Why is the derivative of the activation functions in neural networks important? I'm new to NN. I am trying to understand some of its foundations. One question that I have is: why the derivative of an activation function is important (not the function itself), and why it's the derivative which is tied to how the network performs learning?

For instance, when we say a constant derivative isn't good for learning, what is the intuition behind that? Is the activation function somehow like a hash function that needs to well differentiate small variance in inputs?

If what you are asking is what is the intuition for using the derivative in backpropagation learning, instead of an in-depth mathematical explanation:

Recall that the derivative tells you a function's sensitivity to change with respect to a change in its input. A high (absolute) value for the derivative at a certain point means that the function is very steep, and a small change in input may result in a drastic change in its output; conversely, a low absolute value means little change, so not steep at all, with the extreme case that the function is constant when the derivative is zero.

Training a neural network essentially amounts to an optimization problem where one wants to minimize a certain value, in this case the error produced by the network on the given training examples. Backpropagation learning can be viewed as a case of gradient descent (the inverse of hill climbing).

If for a moment we assume that your input is only 2-dimensional (just for illustration, the mathematics of course also work for higher dimensions), you could imagine the error function as a landscape with hills, mountains, valleys, ridges etc. You are standing at a high point and want to get down as far as possible. Gradient descent means that, in discrete steps, you always walk down in the direction that has the steepest slope downwards from where you are currently standing, until you eventually reach a (local) minimum.

In order to determine where that steepest slope is, you need the derivative of the activation function. Basically, you want to sort out how much each unit in your network contributes to an error, and adjust in the direction that contributes the most.

Edit: Regarding constant values for a derivative, in the landscape metaphor it would mean that the gradient is the same no matter where you are, so you'll always go in the same direction and never reach an optimum. However, multi-layer networks with linear activation function are kind of besides the point anyhow when you consider that each cell computes a linear combination of its inputs, which then is again a linear function, so the output of the last layer will ultimately be a linear function of the inputs at the first layer. That is to say, anything you can do with a multi-layer net with linear activation functions, you could also achieve with just a single layer.


What is the relationship between degrees of freedom and the size of the training dataset? I am going through the book Pattern Recognition by Bishop.

At one point he says

  For $M = 9$, the training set error goes to zero, as we might expect because this polynomial contains 10 degrees of freedom corresponding to the $10$ coefficients $w_0, \dots, w_9$, and so can be tuned exactly to the $10$ data points in the training set.

where $M$ is the order of the hypothesis function, and $w$ are the weights of the hypothesis function.

I did not understand how having $10$ degrees of freedom will tune the model EXACTLY to the $10$ data points? Does it mean that whenever we have a number of data points in training set equal to the degrees of freedom, the error will be zero?

When you define a straight line of the form $y=mx+c$, you need 2 points $(x_1,y_1)$ and $(x_2,y_2)$, to solve for the 2 variables $m$ and $c$ (you can easily visualise this graphically). Similarly, a parabola of the form $y=ax^2+bx+c$ will require 3 such points.

Now viewing it as a ML problem, you are given the points and you have to estimate the parameters such that the training error is 0 (Regression). So just like the previous case you have a bunch of $(x_i,y_i)$ and you have to fit a curve whose degree of freedom you have to choose. Here $m,c,a,b$ are all replaced with more generic $w$ called as a parameter

If you have $10$ degree of freedom and $10$ data-points you can solve for the parameters of the model (unambiguous solution i.e only one and one unique solution will exist). Whereas , if the degree of freedom is lower you'll get a solution which may miss one point. For, example if you are given 3 points and ask to fit a straight line through it, you may or may not be able to (depending on collinearity). On the opposite case if you have more degrees of freedom you can get multiple values for a single parameter. Assuming a single extra degree of freedom, you can take a parameter hold it to some fixed value and solve the rest of the equation as mentioned above you get some value for the rest of parameters. Now hold the same parameter at a different fixed value and repeat the same process, you get some different value of the other parameters.

In general, it is easier to view it this way:

If let's say you have 3 degrees of freedom $y=w_2x_1^2+w_1x_1+w_0$ and 3 data-points $(x_1,y_1), (x_2,y_2), (x_3,y_3)$, you can get a system of equation:

$$y_1=w_2x_1^2+w_1x_1+w_0$$
$$y_2=w_2x_2^2+w_1x_2+w_0$$
$$y_3=w_2x_3^2+w_1x_3+w_2$$

Thus you get a system of linear equations (NOTE: $x_i,y_i$ are known here). The only unknowns here are $w_i$'s for which there are multiple methods to solve. You can extend this for $n$ equations.


How to split data into training validation and test set when the number of data in classes varies greatly? I have 5 classes of pictures to classify:

0 -> ~3 200 (~800 initial number before interference and duplication)

1 -> ~9 000 (I reduced from ~90 000)

2 -> ~8 000

3 -> ~3 000

4 -> ~7 200

How to divide the data?

Now I have divided the data giving 2 000 to test and 2 000 to validation set
by taking a fixed number of images (400) from each class. I don't have much knowledge so I don't know if this is a good division of data.
The attached picture shows the results on the test data after about 60 epoch of CNN with 15 layers.

The network continues to overfiting, and the results of validation and test set do not improve.
I know that I could definitely improve my model but I would like to divide the data in some thoughtful and reasonable way. Pictures are spectrograms and are in RGB format.

I would make the distribution of the classes in test and validation sets the same as in the training set (and as in the whole original set). Anyway all your metrics are relative, not absolute and designed to provide reasonable results when classes are not ideally balanced.


What is the meaning of a 2D stride? I know what meaning stride has when it is just an integer number (by which step you should apply a filter to the image). But about (1, 1) or even more dimensional stride?

Your stride will be just an integer number only when doing 1D convolution. When doing 2D convolution you get 2 integers like your example: (1, 1).

When there is more than one dimension to a stride, then each is applied independently in its associated dimension, and all valid combinations that fit in the area or volume should be calculated.

A 2D stride defines a rectangular grid of points where an operation is applied. The first number is distance between grid points along first (x) axis, the second number is distance between grid points along second (y) axis. You can decide to downsample each dimension differently by using different numbers for each dimension.

In TensorFlow is common to see 4D stride parameters, with axes of (example, height, width, channel). Typically you don't want to skip any examples or channels in a normal CNN, even though the framework can do that, so you will see values like (1, 2, 2, 1) used when using stride to downsample just image height and width dimensions between layers.


What kind of algorithm is the Levenberg–Marquardt algorithm? Is a Levenberg–Marquardt algorithm a type of back-propagation algorithm or is it a different category of algorithm?

Wikipedia says that it is a curve fitting algorithm. How is a curve fitting algorithm relevant to a neural net?

In the context of Neural Networks, Backpropagation (with Gradient Descent, to use its full name) and Levengerg Marquardt are both members of the broader family of gradient descent algorithms. Backpropagation itself is not gradient descent, but it does the gradient climbing portion of a broader gradient descent algorithm.

You can imagine the function of a Neural Network as a function from its inputs to its outputs. If you were trying to solve, for example, a regression problem you can imagine this problem in multidimensional space with each training point corresponding to the coordinate provided by the values of its inputs and outputs (each of which represent a dimension). Then the entire training set you have for learning this regression problem become a set of points in this multidimensional space, and the function that your neural network performs is a curve in this multidimensional space. The closer this curve is to the points on the training set, the better it performs at the regression problem which essentially means we can generalize the task of training the neural network to a curve fitting problem.


Did Turing foresee the required capabilities to pass the Turing test? In Section 1.1 of Artificial Intelligence: A Modern Approach, it is stated that a computer which passes the Turing Test would need 4 capabilities, and that these 4 capabilities comprise most of the field of Artificial Intelligence:

natural language processing: to enable it to communicate successfully in English
knowledge representation: to store what it knows and hears
automated reasoning: to use the stored information to answer questions and to draw new conclusions
machine learning: to adapt to new circumstances and to detect and extrapolate patterns

Did Alan Turing discern the requirements for the field of artificial intelligence (the necessary subfields) and purposefully design a test around these requirements, or did he simply design a test that is so general that the subfields which developed within artificial intelligence happen to be what is required to solve it? That is, was he prescient or lucky? Are these Turing's subdivisions, or Peter Norvig's and Stuart Russell's?

If Turing did foresee these 4 requirements, what did he base them on? What principles of intelligence allowed him to predict the requirements for the field?

I find it unlikely that you'll find a firm answer, so I will try my best to guide you towards information which may help you form an opinion either way. Turing had the controversial opinion (which remains controversial today) that:

  Digital computers have often been described as
  mechanical brains. Most scientists probably
  regard this description as a mere newspaper
  stunt, but some do not. One mathematician has
  expressed the opposite point of view to me rather
  forcefully in the words “It is commonly said that
  these machines are not brains, but you and I
  know that they are.” […] I shall give most attention
  to the view which I hold myself, that it is
  not altogether unreasonable to describe digital
  computers as brains.

In essence Turing believed that the digital computers of the time had the capacity to mimic the human brain if programmed correctly. The problem for Turing then, was finding the correct procedures to mimic the brain. Remember the Imitation Game was being used as a means to explore the question "can machines think?" Turing was engaging in an intellectual debate about the nature of "thinking", he was not making an attempt to describe how machines would be taught to think.

Since, both artificial intelligence and machine learning are at their core very similar to "artificial thinking" (as Turing may have called it) it seems necessary that a test designed to decide whether machines can think would also encompass whether machines could be intelligent or whether machines could learn.

Therefore, it seems possible that Turing did not make the question with these capabilities in mind, but rather that these capabilities are what guided Turing to this question as they are so innately tied to thinking.

If you would like to look further into Turing's thought process here is his paper on the imitation game and here is a discussion on the proceedings of a BBC broadcast about "Automatic Calculating Machines" in which Turing was a speaker.


How widely accepted is the definition of intelligence by Marcus Hutter & Shane Legg? I came across several papers by M. Hutter & S. Legg.
Especially this one:
Universal Intelligence: A Definition of Machine Intelligence, Shane Legg, Marcus Hutter
Given that it was published back in 2007, how much recognition or agreement has it received?
Has any other work better formalizing the idea of intelligence been done since?
What is considered current standard on the topic in the field?

My sense is that everyone is pretending Intelligence doesn't have a grounded definition, from which all other definitions arise:

Intelligence is a measure of utility in an action space μ(υ)

It can be a relative measure, in relation to other rational agents, or absolute in relation to solved games (problems).  An action space is any context, and formalized as problems or sets of problems, typically grouped by complexity class.
Hutter and Legg is an explication of this grounded definition which accounts for unlimited contexts (complexity classes/environments) and for increasing utility of a given agent over time (learning/optimization.)  Intelligence itself does not require learning or general applicability, but Hutter & Legg does not refute this, merely grades static intelligence and narrow intelligence as more limited.
Even this is subject to context, as more limited rationality can be more optimal.

The definition of intelligence is grounded because, while the term "intelligence" is a symbol, intelligence itself is function, the strength of which evaluated by a measurement of some result (utility)

It doesn't require defining the function to understand it as a function:  measurement of a result requires mechanism and decision/action.
You will find this natural language definition applies to even emotional intelligence, which relates to the observational capability of the rational agent in context, and allow that rational agent to make more optimal decisions in context.
This is similar in spirit to truth, only grounded in a formal logic context, where it is a condition and result, not an assertion.  By contrast, the the conditionality of truth is often obscured in a natural language context, where is is routinely applied to unvalidatable informal statements, and can even be conflated with the statement itself à la: "this is the truth!"


Why is breadth-first search only optimal when the cost solution is a non-decreasing function? I am learning about searching strategies in AI and I was reading that breadth-first search is only optimal when the cost solution is a non-decreasing function? I am not really sure what this refers to since decreasing search cost should be our goal. Am I missing something?

This is well covered in the corresponding chapters of Russell & Norvig (Ch. 3 & 4). It also depends on the distinction between TREE-SEARCH and GRAPH-SEARCH.

First, note that Breadth-first search also can't handle cost functions that vary between nodes! Breath-first search only cares about the number of moves needed to reach a state, not the total cost of getting there, so if some states are cheaper on a per-move basis, it's not optimal.

Ok, back to the question: basically, the statement you reference applies to Breadth-first search run on a GRAPH-SEARCH problem. In this kind of problem, it is possible to loop back to an earlier state from a later state (like in chess, where you may move your king back and forth).

If the costs for moving between two states are negative, but constant across all the possible moves, then Breadth-first search will not chose to loop between them forever, since it always expands the nearest node to the start by total number of moves, not the cheapest node. Since you could always add one more loop, to make a path shorter, breadth-first search will not find the shortest path (by cost), but will find the shortest path by number of moves instead.


Why don't we use auto-encoders instead of GANs? I have watched Stanford's lectures about artificial intelligence, I currently have one question: why don't we use autoencoders instead of GANs?
Basically, what GAN does is it receives a random vector and generates a new sample from it. So, if we train autoencoders, for example, on cats vs dogs dataset, and then cut off the decoder part and then input random noise vector, wouldn't it do the same job?

In fact, autoencoders are used for generative tasks. Have a look at Tutorial on Variational Autoencoders (VAEs).
The coolest thing about VAE is that abstract features can be easily amplified or suppressed based on extracted vectors from the latent space. Let's imagine a model trained on MNIST to generate digits. If you take two images of the same digit which only vary in thickness, encode both of them and subtract the two vectors, the resulting vector will be a description of thickness in the latent space. Now you can generate an arbitrary digit and incrementally adjust its thickness based on that vector.


Learning Rate Decay and Exploration Rate Decay Should I be decaying the learning rate and the exploration rate in the same manner? What's too slow and too fast of an exploration and learning rate decay? Or is it specific from model to model?

Welcome to AI.SE !
First of all, I'd say that there is a reason to give Learning Rate (LR) and Exploration Rate (ER) the same decay : they play at the same scale (the number of successives batches you'll train your model on). But if I refine the analysis, I would rather say that it's a reason to choose them in the same range, i.e. close to 1, but not specially at the same number.

For LR decay, people often choose it very close to one (which can mean really different things like 0.98 or 0.997), because it plays on a large scale, an you don't want the LR to disappear to brutally.
However, the choice of ER decay can have more variation from model to model. It depends on the initial value of ER (you don't wanna decay fastly ER if you ER is initially low), and also to the "learning speed" of your model : if your model learns efficiently at the beginning, you could want to fastly decrease ER in order to reduce the noise on the action, supposing that you did enough exploration at the beginning (but I think this last opinion is more controversial). You can find an interesting paper here, where the author tries different ER decay and finds out that 0.99 is the best, for CartPole environment.


Why is neural networks being a deterministic mapping not always considered a good thing? Why is neural networks being a deterministic mapping not always considered a good thing?
So I'm excluding models like VAEs since those aren't entirely deterministic. I keep thinking about this and my conclusion is that often times neural networks are used to model things in reality, which often time do have some stochasticity and since neural networks are deterministic if they are not trained on enough examples of the possible variance inputs in relation to outputs can have they cannot generalize well. Are there other reasons this is not a good thing?

Your intuition is right. The main reason why a deterministic function can be undesirable (or even dangerous, as I will explain below with an example) is that we may not have enough data to learn the correct function, so we may end up learning the incorrect one. Right now, no other reason, from a theoretical point of view, comes to my mind, but below I will mention a few applications/cases where a deterministic function may not be desirable.
If we had all data pairs $\{(x_i, y_i)\}$, where $x_i \in \mathcal{X}$ and $y_i = \mathcal{Y}$ are, respectively, an input and output from the unknown function that you want to learn $f: \mathcal{X} \rightarrow \mathcal{Y}$, i.e. $f(x_i) = y_i$, then you could reconstruct $f$: whenever $x_i$ is given, you just need to return $y_i$.
Of course, in reality, we almost never have a large enough (training) dataset to approximate out desired (but usually unknown) function. If we learn only one (deterministic) function, then, in principle, you can catastrophically fail, i.e. your approximation of $f$, denoted as $f_\theta$ (where $\theta$ are the parameters of the neural network or any other model), can produce outputs that are completely wrong.
Let me try to give you a simple example. Let's say that $f$ is defined as follows
$$f: \mathbb{N} \rightarrow \{0, 1\}$$
You are given a training labelled dataset $$D = \{(4, 1), (11, 0), (8, 1), (31, 0), (16, 1), (7, 0) \}.$$
Apparently, our unknown function is defined as
\begin{align}
h_1(x)=
\begin{cases}
1, &x \text{ mod } 2 \equiv 0\\
0, &\text{otherwise}
\end{cases}\tag{1}\label{1}
\end{align}
Given that $D$ is small, your neural network, $f_\theta$, can easily overfit $D$, i.e. learn to output $1$ when $x$ is even and $0$ otherwise.
However, what if $f$ is not that function in equation \ref{1} and we collected just a dataset that doesn't represent $f$ well enough? If you look at $D$ more carefully, you will see that another possible hypothesis for $f$ is the following
\begin{align}
h_2(x)=
\begin{cases}
1, &x \text{ mod } 4 \equiv 0\\
0, &\text{otherwise}
\end{cases}\tag{2}\label{2}
\end{align}
However, given that your neural network can only compute one of these functions at a time, it could compute the wrong one. Let's say that $f_\theta \approx h_1$, then it should produce $1$ when $x = 6$ (an even number). If the correct unknown function was $h_2$, i.e. $f = h_2$, then $f_\theta(6) = 1$ would be wrong (because $6$ is not a multiple of $4$).
Of course, this is just a toy example. However, there are many other cases where this can happen, which may not be desirable, such as healthcare, medicine or self-driving cars, where the wrong prediction can lead to catastrophic consequences, such as the death of a person.
If we maintain a probability distribution over the possible functions that are consistent with the observed data so far, we can (partially) avoid this issue. So, continuing with the example above, this probability distribution over functions should be highly uncertain about $x = 6$, whether it produces $0$ or $1$, because it has never seen the label for $x=6$, so a medical doctor or the human driver could intervene in the case of (high) uncertainty.
For this reason, in the last decade, people have started to incorporate uncertainty estimation in neural networks. Neural networks that model uncertainty (to some degree) are often called Bayesian neural networks (BNNs), and there are different approaches (such as variational BNNs, MC dropout or Monte Carlo-based approaches). If you are interested in this topic, the paper Weight Uncertainty in Neural Network (2015) is a good start, especially if you are already familiar with VAEs. Given that this is a very new research area, the current solutions are still not very satisfactory. For example, you can find examples in the literature that report that MC dropout can produce very bad estimates of uncertainty (even in my master's thesis I have observed and thus concluded that this is the case), i.e. they can be highly certain when they should be highly uncertain.


Should I use anthropomorphic language when discussing AI? The English Language is not well-suited to talking about artificial intelligence, which makes it difficult for humans to communicate to each other about what an AI is actually "doing". Thus, it may make more sense to use "human-like" terms to describe the actions of machinery, even when the internal properties of the machinery do not resemble the internal properties of humanity.

Anthropomorphic language had been used a lot in technology (see the Hacker's Dictionary definition of anthropomorphization, which attempts to justify computer programmers' use of anthromporhic terms when describing technology), but as AI continues to advance, it may be useful to consider the tradeoffs of using anthropomorphic language in communicating to both technical audiences and non-technical audiences. How can we get a good handle on AI if we can't even describe what we're doing?

Suppose I want to develop an algorithm that display a list of related articles. There are two ways by which I can explain how the algorithm works to a layman:

Very Anthropomorphic - The algorithm reads all the articles on a website, and display the articles that are very similar to the article you are looking at.
Very Technical - The algorithm converts each article into a "bag-of-words", and then compare the "bag-of-words" of each article to determine what articles share the most common words. The articles that share the most words in the bags are the ones that are displayed to the user.

Obviously, #2 may be more "technically correct" than #1. By detailing the implementation of the algorithm, it makes it easier for someone to understand how to fix the algorithm if it produces an output that we disagree with heavily.

But #1 is more readable, elegant, and easier to understand. It provides a general sense of what the algorithm is doing, instead of how the algorithm is doing it. By abstracting away the implementation details of how a computer "reads" the article, we can then focus on using the algorithm in real-world scenarios.

Should I, therefore, prefer to use the anthropomorphic language as emphasized by Statement #1? If not, why not?

P.S.: If the answer depends on the audience that I am speaking to (a non-technical audience might prefer #1, while a technical audience may prefer #2), then let me know that as well.

If clarity is your goal, you should attempt to avoid anthropomorphic language - doing so runs a danger of even misleading yourself about the capabilities of the program.

This is a pernicious trap in AI research, with numerous cases where even experienced researchers have ascribed a greater degree of understanding to a program than is actually merited.

Douglas Hofstadter describes the issue at some length in a chapter entitled "The Ineradicable Eliza Effect and Its Dangers" and there is also a famous paper by Drew McDermot, entitled "Artifical Intelligence meets natural stupidity".

Hence, in general one should make particular effort to avoid anthropomorphism in AI. However, when speaking to a non-technical audience, 'soundbite' descriptions are (as in any complex discipline) acceptable provided you let the audience know that they are getting the simplified version.


Loss randomly changing, incorrect output (even for low loss) when trying to overfit on a single set of input and output I am trying to make a neural network framework from scratch in C++ just for fun, and to test my backpropagation, I thought it would be an easy way to test the functionality if I give it one input - a randomized size 10 vector, and one output: a size 5 vector containing all 1s, and train it a bunch of times to see if the loss will decrease. Essentially trying to make it overfit
The problem is that for each run that I do, the loss either shoots up and goes to nan, or reduces a lot, going to 0.000452084 or other similar small values. However even in the low end of things, my output (which should be close to all 1s, as the "ground truth") is something like:
0.000263654
1e-07
8.55893e-05
1e-07
0.999651

The only close value close to 1 being the last element.
My network consists of the input layer 10 neurons, one 10 neuron dense layer with RELU activation, and another 5 neuron dense layer for output, with SoftMax activation. I am using categorical cross entropy as my loss function, and I am normalizing my gradient by dividing it by the norm of my gradient if it is over 1.0. I initialize my weights to be random values between -0.1 and 0.1
To calculate the gradient of the loss function, I use -groundTruth/predictedOutput. To calculate
the other derivatives, I dot the derivative of that layer with the gradient of the previous layer with respects to its activation function.
Before this problem I was having exploding gradients, which the gradient scaling fixed, however it was very weird that that would even happen on a very small network like this, which could be related to the problem I am currently having. Is the implementation not correct or am I missing something very obvious?
Any ideas about this weird behavior, and where I should look first? I am not sure how to show a minimal reproduceable example as that would require me to paste the whole codebase, but I am happy to show pieces of code with explanation. Any advice welcomed!!

Softmax activation always adds up to 1, because it's designed to deal with probabilities (in problems of classification, those probabilities represent how likely the network thinks an object belongs to a specific class). You can verify that by summing up the numbers of your output layer. So currently your network is trying to do the impossible, to produce output that sums up to 5, instead of 1. Therefore, loss will never become stable. Since you want your output layer to produce all ones, you need to use some other activation, for example, linear. Linear activation does not have the same constraint that Softmax does.


What is the importance of the endocannabinoid system for cognitive function? The endocannabinoid system is a very important function of human biology. Unfortunately, due to the illegality of cannabis, it is a relatively new field of study. I have read a few articles about Google researching the role of dopamine in learning, and according to this article, anandamide (the neurotransmitter that closely resembles tetrahydrocannabinol):

  was found to do a lot more than produce a state of heightened happiness. It’s synthesized in areas of the brain that are important in memory, motivation, higher thought processes, and movement control.

Have any neuroscientists (or any scientists) considered the importance of the endocannabinoid system for cognitive function?

If not, is there any reason this information might or might not be relevant to artificial intelligence?

The release of Adenosine, Dopamine, Endorphin, Endocannabinoids, GABA, Glutamate, Norepinephrine, Oxytocin, Serotonin, and many others into specific regions of the brain are very likely an essential part of both activation tuning of single neurons and neuroplasticity, two essential aspects of organic learning researchers have been and will continue to work to understand.

Most of those I've met in that sector of research are curious about the larger questions of what intelligence and consciousness are, and all of them appear to be interested in discovering how learning systems may be valuable in software engineering contexts.  These overarching questions are difficult to answer and the dive into the detail of learning has resulted in the expansion of ideas presented by Dr. Norbert Wiener in the mid 20th Century at MIT.

How chemical feedback in regions of the brain are secreted, how they disseminate geometrically into organic structures, how they interact with receptors, and what that does to the cell metabolism to produce change in the cell is almost definitely part of the DNA driven design of higher animal learning.  There does not appear to be anything pointless or arbitrary about it.  Adaptation to improve survival is evident and, yes, study is well underway.

Narcotics that interfere with the natural functioning of these organic signaling systems can lead to the inability to adapt in the individuals addicted to them.  That fact is a strong form of evidence that learning depends on these systems.

Oxytocin is another neurotransmitter of interest because its release is associated with what would fit into the higher levels of human thought and motivation on Abraham Maslow's famous hierarchy of needs.  Oxytocin seems to be part of reward signaling for modes of thought like authenticity, compassion, intimacy, wisdom, spirituality, and other human mental capacities and patterns of thought that transcend mere rationalism.  Why is that important?  Because the ability to lay down selfish goals for the good of the community seem to depend largely on the oxytocin system's preemptive ability over mere survival mode neural activity in mammals.

Regarding the cannabinoid receptors, there is a large enough body of media that show a correlation between a pool of successful artists and marijuana use to legitimately wonder whether there is any tie between creativity and the endocannabinoid system.  However in science (and hopefully in technology too), we are careful not to draw conclusions rashly.

It is also possible that this apparent correlation is simply a social phenomenon where the popularity of the artistic products or live performances is mainly because of potential audiences similarly stimulating their receptors with cannabinoids.  For instance, those engaging in LSD trips on stage attracted those also engaging in LSD trips into their audiences two generations ago.  Whether the work of those artists was more creative because of the impact on the serotonin receptors is largely subjective.  How can researchers come to conclusions about what is good performance?

One could analyze the audio to produce a table of notes in a performance and then develop a system to attach a numerical value to several positive quality metrics, study the price of tickets, or count downloads, but the decisions of what to measure and how to aggregate it into a final judgment of excellence is itself necessarily a subjective choice.

Nonetheless, if we place the many misconceptions of popular psychology and the drug culture aside, there is much research into the endocannabinoid system as part of the learning signaling, what machine learning researchers are currently calling reinforcement.  The more general term is, "A non-linear control system's feedback signal," already developed in detail in the 1940s (Behavior, Purpose and Teleology — A Rosenblueth, N Wiener, J Bigelow — Philosophy of Science, 1943 — U Chicago).  Some new trendy name will probably appear in the 2020s.

This article claims that the mammals under test, "exhibited enhanced learning."

Memory in Monoacylglycerol Lipase Knock-out Mice, by Bin Pan, Wei Wang, Peng Zhong, Jacqueline L. Blankman, Benjamin F. Cravatt and Qing-song Liu; Journal of Neuroscience 21 September 2011, 31 (38) 13420-13430; DOI: https://doi.org/10.1523/JNEUROSCI.2075-11.2011

To find many others: https://scholar.google.com/scholar?q=endocannabinoid+learning


Convolutional Neural Networks for different-sized Source and Target CNNs are often used in one of the following scenarios:

A known-sized image is encoded to an intermediate format for later use
An intermediate or precursor format is decoded into a known-sized image
An image is converted into a same-size image

(Usually 3 is done by sticking together 1 and 2.)

Are there any papers dealing with convolutional techniques where the image sizes vary?  Not only would the size of input X differ from input Y, but also input X may differ from output Y. The total amount of variation can probably be constrained by the statistics of the dataset, but knowledge of input X does not grant a priori knowledge of the size of output Y

(Masking is an obvious solution, but I am hoping for something more elegant if research already exists.  The problem domain need not be images.)

Yes actually. There have been quite a few different adaptations to convolutional neural networks to do precisely what you are describing.

Here is an earlier one. See section 3.2.5

Here He et al. create a method known as Spatial Pyramid Pooling(SPP). In this method, you are able construct a fixed length representation regardless of input size by pooling them into "spatial bins" which are proportional to the input size, and thusly do not need to modify input dimensions.

There are a few newer methods that improve on this in various capacities, as usual, your solution will be dependent on the problem and other situation-dependent constraints. I suggest you dig deeper into the literature to find the optimal solution in your case.


When are compiled vs. interpreted languages more optimal in AI? When are interpreted languages more optimal?  When are compiled languages more optimal?  What are the qualities and functions that render the so in relation to various AI methods?

Interpreted languages allow for a faster development cycle, as they don't require time for compilation, and fragments can often be run without having a complete program. They often also have fewer constraints for variable declaration or typing. That means they can be used to quickly scope out a problem and try different solutions.

The drawback is the slower execution speed. But during development this is not a big factor; it only becomes important in a production environment. So one option would be to use an interpreted language during the R&D phase, and then re-implement the algorithm in a compiled language for performance improvements.

Since ML and NNs have become more prevalent in AI, numerical computing has become more important. This is an area where interpreted languages traditionally don't perform too well, so one would use a (compiled) library for, say neural networks, or genetic algorithms, and use 'glue code' to integrate this into a bigger system. The glue code would transform/prepare data and convert this between different formats required by libraries. This is often done in interpreted scripting languages, as they might have to be changed more frequently and are not performance critical.

Apart from development, the type of computation is also key: as mentioned, numerical computing generally works better with compiled code, but interpreted languages often have advantages in symbolic programming. This is why Lisp and Prolog have become popular AI languages, as opposed to Fortran or C.

In an ideal world you would use an interpreted language for development, and then compile this once you're done. However, due to the way these languages work, compilation is often non-trivial.


Are there any anthropological studies involving AI right now? Are there currently any studies to simulate gradual (or sudden) implementation of AIs in the general work force?

As far as I can tell (I've been doing searches here and there on and off since I saw this question a few hours ago) the closest we've gotten to 'simulations' on this is video-games, and to a degree movies, interestingly enough. I.e. entertainment media.

Games like Portal, System Shock (with the AI 'Shodan'), and others give interpretations of what AI systems could be capable of themselves. Mass Effect is more or less entirely based around existential concepts regarding extra-terrestrial, almost primordial AI beings that threaten the earth.

But there's even more to it than the whole 'evil robots taking over the world' aspect. There's the actual implementation of AI in video games, which is where much of this technology first makes contact with the general public.

We have facial-scanners that put you into NBA games, cities full of realistically reacting people (inFamous, Assassin's Creed), and games that learn how you play and adjust the game accordingly (Metal Gear Solid does some of that stuff, as well as being thematically AI-heavy).

Ultimately, we only get things like the iPhone or VR headsets or other major proof-of-concept material only so often, but games are much more frequently implementing the most recent AI advances.

Thus, even though many AI systems are being put into place in the general workforce (many Hospitals now turning to cloud and AI health services, as recent as this week), I don't think you can really go further than video games or movies for 'simulations' or extrapolations like the ones you seem to want.

Analyzing the response to AI developments in games might be the closest thing currently possible. In terms of economic models or anything of that sort, I can find naught.


How to recognize non-circular radial symmetry in images? This is a question about pattern recognition and feature extraction.

I am familiar with Hough transforms, the Fast Radial Transform and variants (e.g., GFRS), but these highlight circles, spheres, etc.

I need an image filter that will highlight the centroid of a series of spokes radiating from it, such as the center of a asterix or the spokes of a bicycle wheel (even if the round wheel is obscured.  Does such a filter exist?

The Hough Transform extended to orthogonal ellipses uses this model, accumulating on $\theta$ for all $\{x, y\}$ with parameter matrix

\begin{Bmatrix}
c_x & c_y \\
r_x & r_y
\end{Bmatrix}

where

$$1 = \dfrac {(x - c_x) \, \cos \theta} {r_x} + \dfrac {(y - c_y) \, \sin \theta} {r_y}$$

The question is looking to detect the normal lines, so any of the several algorithms for the above model can be modified to accumulate on $r$ for all $\{x, y\}$ with parameter matrix

\begin{Bmatrix}
c_x & c_y  \\
r_x & r_y
\end{Bmatrix}

where

$$0 = \dfrac {x - c_x} {r_x} + \dfrac {y - c_y} {r_y}$$

Lines that intersect $(c_x, c_y)$ don't rely on $r_x$ or $r_y$.  However, it may be useful to recognize that, if radially equally spaced, viewing the lines from a position other than one that projects into the plane of the lines at $(c_x, c_y)$ will present a line density that is a function of $\arctan (r_x, r_y)$.


Neural machine translation, that outputs multiple alternative, ambiguous translations? Is there neural machine translation methods, that for one input sentence outputs multiple alternative output sentences in that target language. It is quite possible, that sentence in source language have multiple meanings and it is not desirable that neural network discards some of the meanings if there is no context for disambiguation provided. How multiple outputs can be acommodated into encode-decoder architecture, or different architecture is required?

I am aware of only one work https://arxiv.org/abs/1805.10844 (and one referen herein) but I am still digesting whether their network outputs multiple sentences or whether it just acommodates variations during training phase.

The LSTM seq2seq model typically used for language translation actually can output multiple variations, and does in many implementations.

The decoder stage outputs confidence/probabilities for each word, and it is possible to run the decoder multiple times, taking different samples on each run.

Not only is it possible, but this is actually done by translation engines. A technique called BEAM search is used to manage multiple translations in progress and pick an eventual one to display, based on whether the language model predicts an overall high or low probability for a sentence (this allows for sentences that start with an unusual initial word choice but that are a better choice in context of what is being translated). It would be a minor change to existing algorithms to display a selection of these instead.

Technically you don't need to sample randomly, and can just take the top N probabilities on each step, and recalculate each time on combined probabilities, culling partial sentences that have low probability in favour of continuing more of the high probability sentences.