Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a machine learning model that has been trained through reinforcement learning to select a content item. One of the methods includes receiving first data characterizing a first context in which a first content item may be presented to a first user in a presentation environment; and providing the first data as input to a long-term engagement machine learning model, the model having been trained through reinforcement learning to: receive a plurality of inputs, and process each of the plurality of inputs to generate a respective engagement score for each input that represents a predicted, time-adjusted total number of selections by the respective user of future content items presented to the respective user in the presentation environment if the respective content item is presented in the respective context.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A system comprising: a long-term engagement machine learning model implemented by one or more computers, the long-term engagement machine learning model having been trained through reinforcement learning to: receive a plurality of inputs, each input including data characterizing a respective context in which a respective content item may be presented in a presentation environment to a respective user, and process each of the plurality of inputs to generate a respective engagement score for each input that represents a predicted, time-adjusted total number of selections by the respective user of future content items presented to the respective user in the presentation environment if the respective content item is presented in the respective context; and a subsystem configured to perform operations comprising: receiving first data characterizing a first context in which a first content item may be presented to a first user in the presentation environment; providing the first data as input to the long-term engagement machine learning model to obtain a first engagement score that represents a predicted, time-adjusted total number of selections by the first user of future content items presented to the first user in the presentation environment if the first content item is presented in the first context; and determining, from the first engagement score, whether or not to present the first content item to the first user in the first context. 2. The system of claim 1, wherein determining whether or not to present the first content item to the first user comprises: providing second data characterizing the first context and a null content item as input to the long-term engagement machine learning model, wherein the long-term engagement machine learning model has been trained through reinforcement learning to treat the second data as an indication that no content item is presented to the first user in the first context and to generate a null engagement score that represents a predicted, time-adjusted total number of selections by the first user of future content items presented to the first user in the presentation environment if no content item is presented in the first context. 3. The system of claim 2, wherein determining whether or not to present the first content item to the first user comprises: determining, from the null engagement score and the first engagement score, a predicted impact on user engagement for the first user as a consequence of presenting the first content item in the first context; and determining, using the predicted impact, whether or not to present the first content item to the first user in the first context. 4. The system of claim 3, wherein determining whether or not to present the first content item comprises determining to present the first content item to the first user in the first context only when the predicted impact is greater than a threshold impact value. 5. The system of claim 1, the operations further comprising: in response to determining to present the first content item, providing the first content item for presentation to the first user in the presentation environment or providing an indication to an external system that causes the external system to provide the first content item for presentation to the first user in the presentation environment. 6. The system of claim 1, wherein the long-term engagement machine learning model has been trained through reinforcement learning to determine trained values of parameters of the long-term engagement machine learning model. 7. The system of claim 1, wherein the data characterizing the first context comprises data characterizing content items previously presented to the first user in the presentation environment. 8. The system of claim 1, wherein the presentation environment is a response to a search query submitted by the user, and wherein the data characterizing the context comprises the search query. 9. The system of claim 8, wherein the data characterizing the first context comprises data characterizing content items previously presented to the first user in response to one or more search queries previously submitted by the first user. 10. The system of claim 1, wherein the data characterizing the first context comprises data characterizing a quality of the first content item. 11. The system of claim 1, wherein the data characterizing the first context comprises a predicted likelihood that the first user will select the first content item if the first content item is presented to the first user in the first context. 12. One or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to implement: a long-term engagement machine learning model, the long-term engagement machine learning model having been trained through reinforcement learning to: receive a plurality of inputs, each input including data characterizing a respective context in which a respective content item may be presented in a presentation environment to a respective user, and process each of the plurality of inputs to generate a respective engagement score for each input that represents a predicted, time-adjusted total number of selections by the respective user of future content items presented to the respective user in the presentation environment if the respective content item is presented in the respective context; and a subsystem configured to perform operations comprising: receiving first data characterizing a first context in which a first content item may be presented to a first user in the presentation environment; providing the first data as input to the long-term engagement machine learning model to obtain a first engagement score that represents a predicted, time-adjusted total number of selections by the first user of future content items presented to the first user in the presentation environment if the first content item is presented in the first context; and determining, from the first engagement score, whether or not to present the first content item to the first user in the first context. 13. A method performed by one or more computers, the method comprising: receiving first data characterizing a first context in which a first content item may be presented to a first user in a presentation environment; providing the first data as input to a long-term engagement machine learning model to obtain a first engagement score that represents a predicted, time-adjusted total number of selections by the first user of future content items presented to the first user in the presentation environment if the first content item is presented in the first context, the long-term engagement machine learning model having been trained through reinforcement learning to: receive a plurality of inputs, each input including data characterizing a respective context in which a respective content item may be presented in the presentation environment to a respective user, and process each of the plurality of inputs to generate a respective engagement score for each input that represents a predicted, time-adjusted total number of selections by the respective user of future content items presented to the respective user in the presentation environment if the respective content item is presented in the respective context; and determining, from the first engagement score, whether or not to present the first content item to the first user in the first context. 14. The method of claim 13, wherein determining whether or not to present the first content item to the first user comprises: providing second data characterizing the first context and a null content item as input to the long-term engagement machine learning model, wherein the long-term engagement machine learning model has been trained through reinforcement learning to treat the second data as an indication that no content item is presented to the first user in the first context and to generate a null engagement score that represents a predicted, time-adjusted total number of selections by the first user of future content items presented to the first user in the presentation environment if no content item is presented in the first context. 15. The method of claim 14, wherein determining whether or not to present the first content item to the first user comprises: determining, from the null engagement score and the first engagement score, a predicted impact on user engagement for the first user as a consequence of presenting the first content item in the first context; and determining, using the predicted impact, whether or not to present the first content item to the first user in the first context. 16. The method of claim 15, wherein determining whether or not to present the first content item comprises determining to present the first content item to the first user in the first context only when the predicted impact is greater than a threshold impact value. 17. The method of claim 13, further comprising: in response to determining to present the first content item, providing the first content item for presentation to the first user in the presentation environment or providing an indication to an external system that causes the external system to provide the first content item for presentation to the first user in the presentation environment. 18. The method of claim 13, wherein the data characterizing the first context comprises data characterizing content items previously presented to the first user in the presentation environment. 19. The method of claim 13, wherein the data characterizing the first context comprises data characterizing a quality of the first content item. 20. The method of claim 13, wherein the data characterizing the first context comprises a predicted likelihood that the first user will select the first content item if the first content item is presented to the first user in the first context.