Certain embodiments involve generating personalized recommendations for users by inferring a propensity of each individual user to accept a recommendation. For example, a system generates a personalized user model based on a historical transition matrix that provides state transition probabilities from a general population of users. The probabilities are adjusted based on the propensity for a user to accept a recommendation. The system determines a recommended action for the user to transition between predefined states based on the user model. Once the user has performed an activity that transitions from a current state, the system adjusts a probability distribution for an estimate of the propensity based on whether the activity is the recommended action.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A method, comprising: receiving, in a computing device, a first location of a user, wherein the first location is proximate to a location included in a collection of predefined locations; generating, in the computing device, a personalized user model based on a historical transition matrix that provides location transition probabilities from a general population of users, wherein the probabilities are adjusted in the user model based on a propensity for the user to accept a recommendation; determining, in the computing device, a first recommended location for the user from among the predefined locations by using a recommendation engine configured with the user model; sending, by the computing device, the first recommended location to the user; determining, by the computing device, that the user has moved to a second location; adjusting, by the computing device, a probability distribution for an estimate of the propensity for the user to accept a recommendation based on whether the second location is the first recommended location; and determining, in the computing device, a second recommended location for the user by using the recommendation engine configured with the user model and the adjusted estimate of the propensity for the user to accept a recommendation. 2. The method of claim 1, wherein adjusting the probability distribution for the estimate comprises adjusting the probability distribution using Bayes' rule, wherein using Bayes' rule comprises updating each probability of the probability distribution based on a likelihood function. 3. The method of claim 1, wherein the user model is a primitive of a Partially Observable Markov Decision Process (POMDP), the method further comprising configuring, in the computing device, a hidden state for the POMDP to represent the propensity for the user to accept a recommendation. 4. The method of claim 3, wherein determining the first recommended location comprises solving the POMDP using Thompson sampling, comprising: reducing the POMDP to a finite set of Markov Decision Processes (MDP)s; solving each of the MDPs and caching an optimal action for each possible value of the propensity; sampling a plausible propensity value from the probability distribution for the estimate of the propensity for the user to accept a recommendation; and identifying the first recommended location as the optimal action for the plausible propensity value of the user. 5. The method of claim 1, wherein the recommendation engine is further configured by the user model to maximize a reward function, wherein the reward function indicates a desirability for individual ones of the predefined locations. 6. The method of claim 1, further comprising adjusting, by the computing device, the probability distribution for the estimate each time the recommendation engine determines that the user moves to another location subsequent to another recommended location being sent to the user. 7. The method of claim 1, further comprising storing, in the computing device, the estimate of the propensity for the user to accept a recommendation, wherein subsequent recommendations for the user are initially generated based on the stored estimate of the propensity for the user to accept a recommendation. 8. The method of claim 1, wherein a uniform distribution is initially used for the probability distribution representing the estimate of the propensity for the user to accept a recommendation. 9. A system, comprising: a computing device comprising a memory and a processor; and a recommendation engine stored in the memory and executed by the processor, the recommendation engine comprising logic that, when configured by a personalized user model providing state transition probabilities that account for a propensity for a user to accept a recommendation: receives a first state for the user that is one of a collection of predefined states associated with possible user activities; determines a first recommended action for the user to transition from the first state to a second state of the predefined states based on the user model and the first state; sends the first recommended action to the user; determines that the user has performed a first activity that transitions from the first state; adjusts a probability distribution for an estimate of the propensity for the user to accept a recommendation based on whether the first activity is the first recommended action; and determines a second recommended action for the user by using the recommendation engine configured with the user model and the adjusted estimate of the propensity for the user to accept a recommendation. 10. The system of claim 9, wherein the collection of predefined states comprises different webpages, different geographic locations, or different purchases. 11. The system of claim 9, wherein adjusting the probability distribution for the estimate comprises adjusting the probability distribution using Bayes' rule, wherein using Bayes' rule comprises updating each probability of the probability distribution based on a likelihood function. 12. The system of claim 9, wherein the recommendation engine is further configured by the user model to maximize a reward function, wherein the reward function indicates a desirability for individual ones of the predefined states. 13. The system of claim 9, wherein the recommendation engine further comprises logic that adjusts the probability distribution for the estimate each time the recommendation engine determines that the user transitions to another state subsequent to another recommended action being sent to the user. 14. The system of claim 9, wherein the user model is a primitive of a Partially Observable Markov Decision Process (POMDP), the POMDP is configured with a hidden state to represent the propensity for the user to accept a recommendation. 15. The system of claim 14, wherein determining the first recommended action comprises solving the POMDP using Thompson sampling, comprising: reducing the POMDP to a finite set of Markov Decision Processes (MDP)s; solving each of the MDPs and caching an optimal action for each possible value of the propensity; sampling a plausible propensity value from the probability distribution for the estimate of the propensity for the user to accept a recommendation; and identifying the first recommended action as the optimal action for the plausible propensity value of the user. 16. A non-transitory computer-readable medium embodying a program for generating personalized recommendations for users by inferring a propensity of each individual user to accept a recommendation, the program executable by a processor in a computing device, comprising: code that receives a first location of a user, wherein the first location is proximate to a location included in a collection of predefined locations; code that generates a personalized user model based on a historical transition matrix that provides location transition probabilities from a general population of users, wherein the probabilities are adjusted in the user model based on the propensity for the user to accept a recommendation; code that determines a recommended location for the user from among the predefined locations by using a recommendation engine configured with the user model; code that sends the recommended location to the user; code that determines that the user has moved to a second location; and code that adjusts a probability distribution for an estimate of the propensity for the user to accept a recommendation based on whether the second location is the recommended location. 17. The non-transitory computer-readable medium of claim 16, wherein adjusting the probability distribution for the estimate comprises adjusting the probability distribution using Bayes' rule, wherein using Bayes' rule comprises updating each probability of the probability distribution based on a likelihood function. 18. The non-transitory computer-readable medium of claim 16, wherein the user model is a primitive of a Partially Observable Markov Decision Process (POMDP), the program further comprising code that configures a hidden state for the POMDP to represent the propensity for the user to accept a recommendation. 19. The non-transitory computer-readable medium of claim 18, wherein determining the recommended location comprises solving the POMDP using Thompson sampling, comprising: reducing the POMDP to a finite set of Markov Decision Processes (MDP)s; solving each of the MDPs and caching an optimal action for each possible value of the propensity; sampling a plausible propensity value from the probability distribution for the estimate of the propensity for the user to accept a recommendation; and identifying the recommended location as the optimal action for the plausible propensity value of the user. 20. The non-transitory computer-readable medium of claim 16, wherein the program further comprises code that stores the estimate of the propensity for the user to accept a recommendation, wherein subsequent recommendations for the user are initially generated based on the stored estimate of the propensity for the user to accept a recommendation.