<p>Q-learning is a <a href="http://en.wikipedia.org/wiki/Temporal_difference_learning" rel="nofollow">Temporal difference learning</a> algorithm. For every possible state (board), it learns the value of the available actions (moves). However, it is not suitable for use with <a href="http://ai-depot.com/articles/minimax-explained/" rel="nofollow">Minimax</a>, because the Minimax algorithm needs an evaluation function that returns the value of a position, not the value of an action at that position.</p>

<p>However, temporal difference methods can be used to learn such an evaluation function. Most notably, Gerald Tesauro used the TD(λ) ("TD lambda") algorithm to create <a href="http://en.wikipedia.org/wiki/TD-Gammon" rel="nofollow">TD-Gammon</a>, a human-competitive Backgammon playing program. He wrote an article describing the approach, which you can find <a href="http://www.research.ibm.com/massive/tdl.html" rel="nofollow">here</a>.</p>

<p>TD(λ) was later extended to TDLeaf(λ), specifically to better deal with Minimax searches. TDLeaf(λ) has been used, for example, in the chess program KnightCap. You can read about TDLeaf in <a href="http://arxiv.org/pdf/cs.LG/9901001" rel="nofollow">this paper</a>.</p>
How to use MinMax trees with Q-Learning?
<p>How to use MinMax trees with Q-Learning?</p>

<p>I want to implement a Q-Learning connect four agent and heard that adding MinMax trees into it helps.</p>