Excerpted from:Boyan, Justin. Learning Evaluation Functions for Global Op timization. Ph.D. thesis, Carnegie Mellon University, August 1998. (Available as Technical Report CMU-CS-98-152. ) TD( ) is a popular family of algorithms for approximate policy eva luation in large MDPs. TD( ) works by incrementally updating the value function after each observed transition. It has two major dr awbacks: it makes inefficient use of data, and it requires the user to manu ally tune a stepsize schedule for good performance. For the case of lin ear value function approximations and = 0, the Least-Squares TD (LSTD) algorithm of Bradtke and Barto [5] eliminates all stepsize par ameters and improves data efficiency. This paper extends Bradtke and Barto’s work in three signific ant ways. First, it presents a simpler derivation of the LSTD algorith m. Second, it generalizes from = 0 to arbitrary values of ; at the extreme of = 1, the resulting algorithm is shown to be a practical formulati on of supervised linear regression.