Systems and techniques are provided for determining mutual information with absolute dependency for feature selection. Items may be received from a dataset. Each item may include two random variables. A first random variable may be associated with a first range of discrete values, and a second random variable may be associated with a second range of discrete values. Mutual information between the two random variables may be determined according to one of: I  ( X , Y ) = ∑ x ∈ X   ∑ y ∈ Y    p  ( x , y ) · log  ( p  ( x , y ) p  ( x ) · p  ( y ) )  and I  ( X , Y ) = ∑ x ∈ X   ∑ y ∈ Y    p  ( y ) · log  ( p  ( x , y ) p  ( x ) · p  ( y ) )  , I(X,Y) may be the mutual information between X and Y, x may be a value for X, y may be a value for Y, p(x,y) may be a joint probability distribution function of x and y, p(x) may be a marginal probability distribution function of x, and p(y) may be a marginal probability distribution function of y. The mutual information may be used in a machine learning system to predict a value for one of the random variables for an item for which the value is unknown.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A computer-implemented method performed by a data processing apparatus, the method comprising: receiving at least two items from a dataset, wherein each item comprises two random variables, and wherein a first of the two random variables is associated with a first range of discrete values, and a second of the two random variables is associated with a second range of discrete values; determining mutual information between the two random variables according to at least one of: I  ( X , Y ) = ∑ x ∈ X  ∑ y ∈ Y   p  ( x , y ) · log  ( p  ( x , y ) p  ( x ) · p  ( y ) )    and I  ( X , Y ) = ∑ x ∈ X  ∑ y ∈ Y   p  ( y ) · log  ( p  ( x , y ) p  ( x ) · p  ( y ) )  , wherein I(X,Y) is the mutual information between the first random variable X and the second random variable Y, x is a value from the range of discrete values for the random variable X, y is a value from the range of values for the random variable Y, p(x,y) is a joint probability distribution function of x and y, p(x) is a marginal probability distribution function of x, and p(y) is a marginal probability distribution function of y; and storing the mutual information. 2. The computer-implemented method of claim 1, wherein the first of the two random variables is a feature of the item, and wherein the second of the two random variables is a label for the item. 3. The computer-implemented method of claim 1, further comprising using the mutual information in a machine learning system to predict a value for at least one of the random variables for an item for which the value is unknown. 4. The computer-implemented method of claim 1, wherein the item is an account in an application ecosystem, the first of the two random variables is feature of the account, and the second of the two random variables is a label for the account. 5. The computer-implemented method of claim 4, wherein the feature of the account is one of: whether an application is installed on a device associated with the account and the number of times an application has been used by a device associated with the account, and wherein the label is one of the gender of the owner of the account and the age of the owner of the account. 6. The computer-implemented method of claim 1, wherein each of the at least two items further comprises a third random variable associated with a third range of discrete values, and further comprising: determining the mutual information between the three random variables according to I  ( X , Y ) = ∑ z ∈ Z  ∑ x ∈ X  ∑ y ∈ Y   p  ( y , z ) · log  ( p  ( z ) · p  ( x , y , z ) p  ( x , z ) · p  ( y , z ) )  , wherein I(X,Y) is the mutual information between the two random variables X and Y that is additional to the mutual information between the third random variable Z and the second random variable Y, z is a value from the range of discrete values for the third random variable Z, p(z) is the marginal distribution function of z, p(y,z) is the joint probability distribution function of y and z, p(x,z) is the joint probability distribution function of x and z, and p(x,y,z) is the joint probability distribution function of x, y and z; and storing the mutual information. 7. The computer-implemented method of claim 6, wherein the third random variable is a feature of the item. 8. A computer-implemented method performed by a data processing apparatus, the method comprising: receiving at least two items from a dataset, wherein each item comprises two random variables, and wherein a first of the two random variables is associated with a first continuous function and the second of the two random variables is associated with a second continuous function; determining mutual information between the two random variables according to at least one of: I  ( X , Y ) = ∫ Y  ∫ X   f  ( x , y ) · log  ( f  ( x , y ) f  ( x ) · f  ( y ) )    x   y   and I  ( X , Y ) = ∫ Y  ∫ X   f  ( x ) · log  ( f  ( x , y ) f  ( x ) · f  ( y ) )    x   y , wherein I(X,Y) is the mutual information between the two random variables X and Y, x is a value from the continuous function for the random variable X, y is a value from the continuous function for the random variable Y, f(x,y) is a joint probability density function of x and y, f(x) is a marginal probability density function of x, and f(y) is a marginal probability density function of y; and storing the mutual information. 9. The computer-implemented method of claim 8, wherein a first of the two random variables is a feature of the item, and wherein a second of the two random variables is a label for the item. 10. The computer-implemented method of claim 8, further comprising selecting based on the determined mutual information the first random variable for use by a machine learning system in predicting a value for the second random variable for an item. 11. The computer-implemented method of claim 8, wherein each of the at least two items further comprises a third random variable associated with a third continuous function, and further comprising: determining the mutual information between the three random variables according to I  ( X , Y ) = ∫ Z  ∫ Y  ∫ X   f  ( y , z ) · log  ( f  ( z ) · f  ( x , y , z ) f  ( x , z ) · f  ( y , z ) )    x   y , wherein I(X,Y) is the mutual information between the two random variables X and Y that is additional to the mutual information between the third random variable Z and the random variable Y, z is a value from the continuous third random variable Z, f(z) is the marginal density function of z, f(y,z) is the joint probability density function of y and z, f(x,z) is the joint probability density function of x and z, and f(x,y,z) is the joint probability density function of x, y and z; and storing the mutual information. 12. The computer-implemented method of claim 13, wherein the third random variable is a feature of the item. 13. A computer-implemented method performed by a data processing apparatus, the method comprising: receiving a dataset comprising items, wherein the items comprise features and labels, each feature comprising a value from a range of discrete values associated with the feature and each label comprising a value from a range of discrete values associated with the label, and wherein at least one feature and one label are common to at least two items in the dataset; determining mutual information between each of the labels and each of the features for the items in the dataset, wherein the mutual information is at least one of mutual information with absolute dependency, accuracy optimized mutual information with absolute dependency, and accuracy optimized adapted conditional mutual information with absolute dependency; and storing the mutual information. 14. The computer-implemented method of claim 13, wherein mutual information with absolute dependency is determined according to: I  ( X , Y ) = ∑ x ∈ X  ∑ y ∈ Y   p  ( x , y ) · log  ( p  ( x , y ) p  ( x ) · p  ( y ) )  , wherein I(X,Y) is the mutual information between a feature X and a label Y, x is a value from the range of discrete values for the feature X, y is a value from the range of values for the label Y, p(x,y) is a joint probability distribution function of x and y, p(x) is a marginal probability distribution function of x, and p(y) is a marginal probability distribution function of y. 15. The computer-implemented method of claim 13, wherein accuracy optimized mutual information with absolute dependency is determined according to: I  ( X , Y ) = ∑ x ∈ X  ∑ y ∈ Y   p  ( y ) · log  ( p  ( x , y ) p  ( x ) · p  ( y ) )  , wherein I(X,Y) is the mutual information between a feature X and a label Y, x is a value from the range of discrete values for the feature X, y is a value from the range of values for the label Y, p(x,y) is a joint probability distribution function of x and y, p(x) is a marginal probability distribution function of x, and p(y) is a marginal probability distribution function of y. 16. The computer-implemented method of claim 13, wherein accuracy optimized adapted conditional mutual information with absolute dependency is determined according to: I  ( X , Y ) = ∑ z ∈ Z  ∑ x ∈ X  ∑ y ∈ Y   p  ( y , z ) · log  ( p  ( z ) · p  ( x , y , z ) p  ( x , z ) · p  ( y , z ) )  , wherein I(X,Y) is the mutual information between a feature X and a label Y that is additional to the mutual information between a feature Z and the label Y, x is a value from the range of discrete values for the feature X, y is a value from the range of values for the label Y, z is a value from the range of discrete values for the feature Z, p(z) is the marginal distribution function of z, p(y,z) is the joint probability distribution function of y and z, p(x,z) is the joint probability distribution function of x and z, and p(x,y,z) is the joint probability distribution function of x, y and z. 17. The computer-implemented method of claim 13, wherein the items are accounts in an application ecosystem. 18. The computer-implemented method of claim 17, wherein at least one of the features is whether an application is installed on a device associated with an account. 19. The computer-implemented method of claim 17, wherein at least one of the labels is the gender of the account owner. 20. The computer-implemented method of claim 17, further comprising selecting based on the determined mutual information at least one feature for use by a machine learning system in predicting a value for at least one label for an item. 21. A computer-implemented system for determining mutual information comprising: a storage comprising mutual information and a dataset, wherein the dataset comprises items, and the items comprises features and labels, each feature and each label associated with a value; and a feature selector adapted to receive the dataset, and generate the mutual information for the features and labels of the items in the dataset according to at least one of wherein the mutual information is at least one of mutual information with absolute dependency, accuracy optimized mutual information with absolute dependency, and accuracy optimized adapted conditional mutual information with absolute dependency. 22. The computer-implemented system of claim 21, wherein the feature selector is further adapted to select based on the mutual information at least one feature for use in predicting a value for a label for an item. 23. A system comprising: one or more computers and one or more storage devices storing instructions which are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising: receiving at least two items from a dataset, wherein each item comprises two random variables, and wherein a first of the two random variables is associated with a first range of discrete values, and a second of the two random variables is associated with a second range of discrete values; determining mutual information between the two random variables according to at least one of: I  ( X , Y ) = ∑ x ∈ X  ∑ y ∈ Y   p  ( x , y ) · log  ( p  ( x , y ) p  ( x ) · p  ( y ) )    and I  ( X , Y ) = ∑ x ∈ X  ∑ y ∈ Y   p  ( y ) · log  ( p  ( x , y ) p  ( x ) · p  ( y ) )  , wherein I(X,Y) is the mutual information between the first random variable X and the second random variable Y, x is a value from the range of discrete values for the random variable X, y is a value from the range of values for the random variable Y, p(x,y) is a joint probability distribution function of x and y, p(x) is a marginal probability distribution function of x, and p(y) is a marginal probability distribution function of y; and storing the mutual information. 24. The system of claim 23, wherein each of the at least two items further comprises a third random variable associated with a third range of discrete values, and wherein the instructions further cause the one more computers to perform operations further comprising: determining the mutual information between the three random variables according to I  ( X , Y ) = ∑ z ∈ Z  ∑ x ∈ X  ∑ y ∈ Y   p  ( y , z ) · log  ( p  ( z ) · p  ( x , y , z ) p  ( x , z ) · p  ( y , z ) )  , wherein I(X,Y) is the mutual information between the two random variables X and Y that is additional to the mutual information between the third random variable Z and the second random variable Y, z is a value from the range of discrete values for the third random variable Z, p(z) is the marginal distribution function of z, p(y,z) is the joint probability distribution function of y and z, p(x,z) is the joint probability distribution function of x and z, and p(x,y,z) is the joint probability distribution function of x, y and z; and storing the mutual information.