Embodiments are disclosed for providing a machine-generated response (e.g., answer) to an input (e.g., question) based on long-term memory information. A method according to some embodiments include receiving an input; converting the input into an input feature vector in an internal feature representation space; updating a memory data structure by incorporating the input feature vector into the memory data structure; generating an output feature vector in the internal feature representation space, based on the updated memory data structure and the input feature vector; converting the output feature vector into an output object; and providing an output based on the output object as a response to the input.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A method for providing a machine-generated response to an input, comprising: receiving, by a computing device, an input; converting, by the computing device, the input into an input feature vector in an internal feature representation space; updating, by the computing device, a memory data structure by incorporating the input feature vector into the memory data structure; generating, by the computing device, an output feature vector in the internal feature representation space, based on the updated memory data structure and the input feature vector; converting, by the computing device, the output feature vector into an output object; and providing, by the computing device, an output based on the output object as a response to the input. 2. The method of claim 1, wherein the input includes a character, a word, a text, a sentence, an image, an audio, a video, a user interface instruction, a computer-generated action, or a question; and wherein the output includes a character, a word, a text, a sentence, an image, an audio, a video, a user interface instruction, a computer-generated action, or an answer to the question. 3. The method of claim 1, wherein the memory data structure includes an array of objects, and at least one of the objects includes information relevant to the input. 4. The method of claim 1, wherein the memory data structure includes an array of vectors or an array of textual strings, and at least one of the vectors or textual strings includes information relevant to the input. 5. The method of claim 1, wherein the memory data structure includes multiple memory slots, at least some of the memory slots storing vectors including information relevant to the input. 6. The method of claim 1, wherein the step of updating a memory data structure comprises: storing the input feature vector in a next available memory slot in the memory data structure. 7. The method of claim 1, wherein the step of generating an output feature vector comprises: identifying one or more supporting memory entries in the memory data structure as being relevant to the input feature vector; and generating the output feature vector based on the input feature vector and the identified supporting memory entries. 8. The method of claim 1, wherein the step of generating an output feature vector further comprises: using a first relevancy function to determine relevancy scores between the input feature vector and memory entries in the memory data structure; and selecting a first memory entry having a highest relevancy score as a first supporting memory entry. 9. The method of claim 8, wherein the step of generating an output feature vector further comprises: using the first relevancy function to determine relevancy scores between a vector sum of the input feature vector and the first supporting memory entry and other memory entries in the memory data structure; selecting a second memory entry having a highest relevancy score as a second supporting memory entry; and generating the output feature vector based on a vector sum of the input feature vector and the first and second supporting memory entries. 10. The method of claim 8, wherein the first relevancy function includes a first weight matrix, and the first weight matrix includes elements that are determined by a training process using pairs of questions and known answers. 11. The method of claim 9, further comprising: training the elements of the first weight matrix of the first relevancy function using a stochastic gradient descent process, wherein during each step, the stochastic gradient descent process samples memory entries in the memory data structure that are not the first or second supporting memory entry. 12. The method of claim 10, wherein the first weight matrix has a dimension determined based on a total number of words in a predetermined dictionary. 13. The method of claim 1, wherein the step of converting the output feature vector comprises: using a second relevancy function to determine relevancy scores between a vector sum of the input feature vector and one or more supporting memory entries and words in a predetermined dictionary; and generating the output textual message including a selected word having a highest relevancy score from the predetermined dictionary. 14. The method of claim 13, wherein the second relevancy function includes a second weight matrix, and the second weight matrix includes elements that are determined by a training process using pairs of questions and known answers. 15. The method of claim 14, further comprising: training the elements of the second weight matrix of the second relevancy function using a stochastic gradient descent process, wherein during each step, the stochastic gradient descent process samples words in the predetermined dictionary that are not the selected word having the highest relevancy score. 16. The method of claim 1, wherein the step of converting the output feature vector further comprises: using a recurrent neural network to generate a sentence as the output object based on the input feature vector and one or more supporting memory entries. 17. A non-transitory computer-readable storage medium storing instructions for providing a machine-generated response to an input, the instructions comprising: receiving, by a computer, an input; converting, by the computer, the input into an internal feature input representation; updating, by the computer, an array of memory objects using the internal feature input representation, the memory objects including supporting information for the textual question; generating, by the computer, an internal feature output representation based on the updated array of memory objects and the internal feature input representation; and decoding, by the computer, the internal feature output representation to generate a response to the input. 18. The non-transitory computer-readable storage medium of claim 17, wherein the instructions further comprise: monitoring utility status of the memory objects of the array, the utility status indicating numbers of times that the memory objects are selected as supporting memories for generating internal feature output representations; selecting, based on the utility status, a memory object having a least number of times as a supporting memory for generating internal feature output representations; and overwriting the selected memory object with new memory information. 19. The non-transitory computer -readable storage medium of claim 17, wherein the instructions for generating an internal feature output representation comprises instructions for: computing, based on a relevancy function, relevancy scores between the internal feature input representation and the memory objects of the array, the relevancy function including parameters that are trained using pairs of questions and known answers; selecting a memory object that has a highest relevancy score as a first supporting memory; and generating the internal feature output representation based on the first supporting memory and the internal feature input representation. 20. The non-transitory computer -readable storage medium of claim 17, wherein the instructions for generating an internal feature output representation further comprises instructions for: constructing multiple buckets corresponding to multiple words in a predetermined dictionary; assigning the memory objects into buckets corresponding to words that the memory objects include; assigning the internal feature input representation into at least one particular bucket corresponding to a word that the internal feature input representation includes; computing, based on a relevancy function, relevancy scores between the internal feature input representation and the memory objects assigned to the particular bucket, the relevancy function including parameters that are trained using pairs of questions and known answers; selecting a memory object within the particular bucket that has a highest relevancy score as a first supporting memory; and generating the internal feature output representation based on the first supporting memory and the internal feature input representation. 21. The non-transitory computer -readable storage medium of claim 17, wherein the instructions for generating an internal feature output representation further comprises instructions for: training an embedding matrix of a relevancy scoring function using pairs of questions and known answers, the embedding matrix having a dimension corresponding to a number of words in a predetermined dictionary; clustering the words in the predetermined dictionary into multiple buckets corresponding to word clusters, by running a vector quantization on vectors of the embedding matrix; assigning the memory objects into the buckets corresponding to word clusters of which the memory objects include at least one word; assigning the internal feature input representation into at least one particular bucket corresponding to a word cluster of which the internal feature input representation includes at least one word; computing, based on a relevancy function, relevancy scores between the internal feature input representation and the memory objects assigned to the particular bucket, the relevancy function including parameters that are trained using pairs of questions and known answers; selecting a memory object within the particular bucket that has a highest relevancy score as a first supporting memory; and generating the internal feature output representation based on the first supporting memory and the internal feature input representation. 22. The non-transitory computer -readable storage medium of claim 17, wherein indices of the memory objects of the array follow a sequence consistent with write timestamps of the memory objects. 23. The non-transitory computer -readable storage medium of claim 17, wherein the instructions for generating an internal feature output representation further comprises instructions for: computing, based on a triples relevancy function, a relevancy score between the internal feature input representation and a first memory object and a second memory object of the array, the relevancy score of the triples relevancy function determining whether the first memory object is preferable over the second memory object or the second memory object is preferable over the first memory object; selecting a memory object that is preferable over a remainder of the memory objects of the array as a first supporting memory, based on the relevancy scores of the triples relevancy functions; and generating the internal feature output representation based on the first supporting memory and the internal feature input representation. 24. The non-transitory computer -readable storage medium of claim 17, wherein the input includes a character, a word, a text, a sentence, an image, an audio, a video, a user interface instruction, a computer-generated action, or a question; and wherein the output includes a character, a word, a text, a sentence, an image, an audio, a video, a user interface instruction, a computer-generated action, or an answer to the question. 25. A computing system, comprising: a memory data structure including an array of memory objects; an input feature map component configured to convert an input into an input feature vector in an internal feature representation space; a memory update component configured to update the memory data structure by incorporating the input feature vector into the memory data structure; an output feature map component configured to identify one or more supporting memory objects that are relevant to the input feature vector among the memory objects of the updated memory data structure, and generate an output feature vector in the internal feature representation space based on the input feature vector and the one or more supporting memory objects; and a response component configured to convert the output feature vector into an output responding to the input. 26. The computing system of claim 24, further comprising: an output interface configured to present the output as an answer responding to a question presented by the input. 27. The computing system of claim 24, wherein the memory update component is further configured to store the input feature vector in a next available memory slot of the memory data structure, the memory data structure including multiple memory slots for storing memory objects. 28. The computing system of claim 24, wherein the output feature map component is further configured to: determine, using a first relevancy function, relevancy scores between the input feature vector and memory objects in the memory data structure; select a memory object having a highest relevancy score as a first supporting memory object; and generate the output feature vector by combining the input feature vector and the first supporting memory object; wherein the first relevancy function includes an embedding matrix, the embedding matrix including matrix elements that are determined by a training process using pairs of questions and known answers. 29. The computing system of claim 24, wherein the response component is further configured to: determine, using a second relevancy function, relevancy scores between the output feature vector and words in a predetermined dictionary; and generating the output textual answer including a selected word having a highest relevancy score among the words in the predetermined dictionary. 30. The computing system of claim 24, further comprising: a hashing component configured to: cluster words in a predetermined dictionary into multiple buckets, by running a vector quantization on vectors of an embedding matrix; and assign the memory objects of the memory data structure into the buckets of which the memory objects include at least one word; and the output feature map component is further configured to: determine, using a first relevancy function, relevancy scores between the input feature vector and memory objects in a bucket of which the input feature vector includes at least one word; select a memory object having a highest relevancy score as a first supporting memory object; and generate the output feature vector by combining the input feature vector and the first supporting memory object. 31. The computing system of claim 24, wherein the output feature map component is further configured to: determine, based on a triples relevancy function, a relevancy score between the input feature vector and a first memory object and a second memory object in the memory data structure, the relevancy score of the triples relevancy function determining whether the first memory object is preferable over the second memory object or the second memory object is preferable over the first memory object; selecting a memory object that is preferable over a remainder of the memory objects of the array as a first supporting memory, based on the relevancy scores of the triples relevancy functions; and generating the output feature vector based on the first supporting memory and the input feature vector.