Embodiments are disclosed for predicting a response (e.g., an answer responding to a question) using an end-to-end memory network model. A computing device according to some embodiments includes embedding matrices to convert knowledge entries and an inquiry into feature vectors including the input vector and memory vectors. The device further execute a hop operation to generate a probability vector based on an input vector and a first set of memory vectors using a continuous weighting function (e.g., softmax), and to generate an output vector as weighted combination of a second set of memory vectors using the elements of the probability vector as weights. The device can repeat the hop operation for multiple times, where the input vector for a hop operation depends on input and output vectors of previous hop operation(s). The device generates a predicted response based on at least the output of the last hop operation.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A computing device, comprising: a feature extraction component including: a first memory embedding matrix to convert a set of knowledge entries into a first set of memory vectors, a second memory embedding matrix to convert the set of knowledge entries into a second set of memory vectors, and an inquiry embedding matrix to convert an inquiry into an input vector; a memory hop component to conduct one or more hop operations, each individual hop operation includes: determining an input vector for the individual hop operation, generating a probability vector based on the input vector and the first set of memory vectors, generating an output vector as a weighted combination of the second set of memory vectors, weights of the weighted combination being elements of the probability vector; and a prediction component to determine a machine-generated response to the inquiry based on the output vector of a last hop operation among the one or more hop operations. 2. The computing device of claim 1, wherein the set of knowledge entries are a set of sentences, the inquiry is a question seeking an answer based on the sentences, and the machine-generated response is a machine-predicted answer to the question. 3. The computing device of claim 1, wherein the set of knowledge entries are words within a text sequence, the inquiry is to seek a prediction of a next word for the text sequence, and the machine-generated response is a machine-prediction of the next word. 4. The computing device of claim 1, further comprising: a training component to train matrices of the feature extraction component and the prediction component using a training data set including sample knowledge entries, sample inquiry and sample responses. 5. The computing device of claim 4, wherein the training component needs no supervision on picking knowledge entries or memory vectors that are relevant to the inquiry, during the training of the matrices of the feature extraction component and the prediction component. 6. The computing device of claim 1, wherein the prediction component determines the machine-generated response to the inquiry, by steps of: combining the output vector and the input vector of the last hop operation; feeding the combination of the output vector and the input vector into a prediction matrix; and calculating a continuous weighting function of a result of the prediction matrix to produce the machine-generated response to the inquiry. 7. The computing device of claim 1, wherein the individual hop operation generates the probability vector by: feeding a dot product of the input vector and each memory vector of the first set of memory vectors into a continuous weighting function to produce an element of the probability vector; and determining the probability vector including the elements produced by the continuous weighting function corresponding to the memory vectors of the first set. 8. The computing device of claim 7, wherein each element of the probability vector indicates a level of relevancy between the inquiry and a knowledge entry corresponding to the element. 9. The computing device of claim 7, wherein the continuous weighting function is a softmax function, the softmax function allows back-propagation of training error signal through the softmax function to the inquiry and the knowledge entries during training of the feature extraction component and the prediction component. 10. The computing device of claim 1, wherein the memory hop component determines the input vector for the individual hop operation by: in an event that the individual hop operation is a first hop operation, determining that the input vector converted from the inquiry is the input vector for the first hop operation; and in an event that the individual hop operation is not a first hop operation, determining the input vector for the individual hop operation as a combination of an input vector and an output vector for another hop operation that is immediately prior to the individual hop operation. 11. The computing device of claim 1, wherein the first memory embedding matrix for the one or more hop operations share the same parameters, and the second memory embedding matrix for the one or more hop operations share the same parameters. 12. The computing device of claim 11, wherein in an event that the individual hop operation is not a first hop operation, determining the input vector for the individual hop operation as a sum of an output vector of another hop operation and an input vector for the other hop operation updated by a linear mapping matrix, the other hop operation being immediately prior to the individual hop operation. 13. The computing device of claim 1, wherein instances of the first memory embedding matrix for each of the hop operations have different parameters, instances of the second memory embedding matrix for each of the hop operations have different parameters; and wherein the individual hop operation further includes: converting the set of knowledge entries into a first set of memory vectors for the individual hop operation, using the instance of the first memory embedding matrix corresponding to the individual hop operation; and converting the set of knowledge entries into a second set of memory vectors for the individual hop operation, using the instance of the second memory embedding matrix corresponding to the individual hop operation. 14. The computing device of claim 13, wherein the instance of the first memory embedding matrix for the individual hop operation share the same parameters as an instance of the second memory embedding matrix for another hop operation that is immediately prior to the individual hop operation. 15. The computing device of claim 14, wherein for a first hop operation, the instance of the first memory embedding matrix share the same parameters with the inquiry embedding matrix; and for a last hop operation, the instance of the second memory embedding matrix share the same parameters with a prediction matrix used by the prediction component to determine the machine-generate response. 16. A method for providing a machine-generated answer to an inquiry, comprising: receiving an inquiry by a computing device, the inquiry seeking an answer based on a set of knowledge sentences; converting, by the computing device, the inquiry into an input vector; conducting a hop operation by using the input vector as input of the hop operation, the hop operation includes: generating a probability vector by a continuous weighting function based on the input vector for the hop operation and an input memory representation of the knowledge sentences, each element of the probability vector indicating a level of relevancy between the inquiry and a corresponding knowledge sentence, and generating an output vector as a weighted combination of an output memory representation of the knowledge sentences, the elements of the probability vector being weights of the weighted combination; and predicting the answer sought by the inquiry based on the output vector. 17. The method of claim 16, further comprising: conducting an additional hop operation by using an updated input vector as input of the additional hop operation, the updated input vector being a combination of the output vector and the input vector of the prior hop operation; the additional hop operation including steps of: generating an updated probability vector by the continuous weighting function based on the updated input vector and the input memory representation of the knowledge sentences; and generating an updated output vector for the additional hop operation as a weighted combination of the output memory representation of the knowledge sentences, the elements of the updated probability vector being weights of the weighted combination. 18. The method of claim 17, further comprising: repeating the additional hop operation for one or more times, wherein an updated input vector for an individual hop operation is a combination of a prior input vector and a prior output vector for another hop operation prior to the individual hop operation. 19. The method of claim 17, wherein the step of predicting the answer comprises: passing a combination of the updated output vector and the updated input vector for a last hop operation through an answer prediction matrix; and predicting the answer sought by the inquiry by calculating a continuous weighting function of an output of the answer prediction matrix. 20. The method of claim 16, wherein the step of converting the inquiry comprises: converting, by the computing device, the inquiry into an input vector by feeding the inquiry to an inquiry embedding matrix; and wherein the method further comprises: converting the set of the knowledge sentences into a first set of memory vectors under the input memory representation, by feeding the set of the knowledge sentences into a first memory embedding matrix; and converting the set of the knowledge sentences into a second set of memory vectors under the output memory representation, by feeding the set of the knowledge sentences into a second memory embedding matrix. 21. The method of claim 20, wherein the continuous weighting function is a softmax function of a dot product of the input vector and a memory vector of the first set corresponding to a knowledge sentence, the softmax function allowing back-propagation of training error signal through the softmax function to the inquiry during training of the inquiry embedding matrix, the first memory embedding matrix and the second memory embedding matrix. 22. The method of claim 20, further comprising: training parameters of the inquiry embedding matrix, the first memory embedding matrix and the second memory embedding matrix by minimizing differences between sample answers and predicted answers that are predicted based on corresponding sample inquiries. 23. The method of claim 20, further comprising: training parameters of the inquiry embedding matrix, the first memory embedding matrix and the second memory embedding matrix by removing the continuous weighting function from each hop operation; and in response to an event that a validation loss stops decreasing, further training the parameters of the inquiry embedding matrix, the first memory embedding matrix and the second memory embedding matrix by re-inserting the continuous weighting function into each hop operation. 24. The method of claim 20, wherein the first memory embedding matrix for an individual hop operation is the same matrix as the second memory embedding matrix for another hop operation immediately prior to the individual hop operation. 24. The method of claim 20, wherein the first memory embedding matrices for multiple hop operations have the same parameters, and the second memory embedding matrices for the multiple hop operations have the same parameters. 25. The method of claim 16, wherein the input memory representation of the knowledge sentences includes encoded information that relates to positions of words within the knowledge sentences. 26. A non-transitory machine-readable storage medium comprising a program containing a set of instructions for providing a machine-generated prediction of a next word in a text sequence, the procedures comprising: receiving a text sequence including multiple words; determining a constant vector as an input vector; conducting a hop operation with the input vector, the hop operation including: converting the words of the text sequence into a first set of memory vectors, by feeding the words to a first memory embedding function; generating a probability vector by calculating a continuous weighting function of a dot product of the input vector and each memory vector of the first set, each element of the probability vector indicating a level of relevancy between the next word to be predicted and a corresponding word from the text sequence, converting the words of the text sequence into a second set of memory vectors, by feeding the words to a second memory embedding function; and generating an output vector as a weighted combination of the second set of memory vectors, the elements of the probability vector being weights of the weighted combination; and predicting the next word for the text sequence based on the output vector. 27. The non-transitory machine-readable storage medium of claim 26, wherein the procedures further comprise: repeating the hop operation for multiple times, wherein the input vector for a particular hop operation is a combination of a prior input vector and a prior output vector for a prior hop operation that is conducted immediately prior to the particular hop operation. 28. The non-transitory machine-readable storage medium of claim 27, wherein the step of predicting the next word comprises: passing a combination of the input vector and the output vector for a last hop operation into a final weight matrix; and generating a prediction of the next word for the text sequence, by feeding an output of the final weight matrix to a continuous weighting function. 29. The non-transitory machine-readable storage medium of claim 26, wherein the step of converting the words of the text sequence into a first set of memory vectors further comprises: converting the words of the text sequence into a first set of memory vectors, by feeding the words as parameters to the first memory embedding function and adding a row of a special matrix that encodes temporal information of the text sequence to each output of the first memory embedding function. 30. The non-transitory machine-readable storage medium of claim 26, wherein the procedures further comprise: training parameters of matrices using a language modeling training set of text sequences and next words; wherein the training of the parameters of the matrices does not require supervision on selecting relevant words from the text sequences of the language modeling training set.