Recurrent neural networks are powerful tools for handling incomplete data problems in machine learning thanks to their significant generative capabilities. However, the computational demand for algorithms to work in real time applications requires specialized hardware and software solutions. We disclose a method for adding recurrent processing capabilities into a feedforward network without sacrificing much from computational efficiency. We assume a mixture model and generate samples of the last hidden layer according to the class decisions of the output layer, modify the hidden layer activity using the samples, and propagate to lower layers. For an incomplete data problem, the iterative procedure emulates feedforward-feedback loop, filling-in the missing hidden layer activity with meaningful representations.
Please help me write a proper abstract based on the patent claims. CLAIM: 1-5. (canceled) 6. A computer implemented method for recurrent data processing, comprising the steps of: computing activity of multiple layers of hidden layer nodes in a feed forward neural network, given an input data instance, forming memories of hidden layer activities, utilizing clustering and filtering methods, as a training phase in a recurrent processing, finding memories that are closest to the presented test data instance according to the class decision of the feedforward network, and imputing the test data hidden layer activity with computed closest memories in an iterative fashion, wherein the step of forming memories of hidden layer activities, utilizing clustering and filtering methods, as a training phase in a recurrent processing further comprises the substeps of: computing hidden layer activities of every training data instance, then low-pass filtering and stacking the hidden layer activities in a data structure, keeping a first and second hidden layer activity memory, indexed with the class label, forming both class specific and class independent duster centers as quantized memories of training data's second hidden layer activity, via k-means clustering, using each class data separately; or using all the data together depending on a choice of class specificity, keeping quantized second hidden layer memories, indexed with class labels or non-indexed, depending on the class specificity choice, training a cascade of classifiers for enabling multiple hypotheses generation of a network, via utilizing a subset of the data as the training data, keeping a classifier memory, indexed with the set of data used during training, wherein the step of finding memories that are closest to the presented test data instance according to the class decision of the feedforward network, and imputing the test data hidden layer activity with computed closest memories in an iterative fashion further comprises the substeps of determining the first, second and third class label choices of the neural network as multiple hypotheses, via a cascaded procedure utilizing a sequence of classifier decisions, computing a set of candidate samples for the second layer, that are closest (Euclidian) hidden layer memories to the test data's second hidden layer activity, using the multiple hypotheses class decisions of the network and a corresponding memory database; then assigning the second hidden layer sample as one of the candidate hidden layer memories, via max or averaging operations depending on the choice of multi-hypotheses competition, merging the second hidden layer sample with the test data hidden layer activity via weighted averaging operation, creating an updated second hidden layer activity, using the updated second hidden layer activity to compute the closest (Euclidian) first hidden layer memory, and assigning as the first hidden layer sample, merging the first hidden layer sample with the test data first hidden layer activity via weighted averaging operation, creating an updated first hidden layer activity, computing the feedforward second hidden layer activity from updated first hidden layer activity, and merging this feedforward second hidden layer activity with updated second hidden layer activity, via weighted averaging operation, repeating these steps for multiple iterations starting from the step of determining the first, second and third class label choices of the neural network as multiple hypotheses, via a cascaded procedure utilizing a sequence of classifier decisions, and using the output of step of computing the feedforward second hidden layer activity from updated first hidden layer activity, and merging this feedforward second hidden layer activity with updated second hidden layer activity, via weighted averaging operation in the beginning of the next iteration. 7. A computer implemented method according to claim 6 for enabling a feedforward network to mimic a recurrent neural network via making a class decision at the output layer of feedforward neural network, and selecting an appropriate memory to estimate selected model's (class decision) hidden layer activities, then inserting the selected memory to the hidden layer activity as if it is a feedback from a higher layer network in classical recurrent networks.