<p>If you skip reinventing the wheel, here is what have done to solve this problem:</p>

<blockquote>
  <p>The model is a convolutional neural network, trained with a variant of
  Q-learning, whose input is raw pixels and whose output is a value
  function estimating future rewards. We apply our method to seven Atari
  2600 games from the Arcade Learning Environment, with no adjustment of
  the architecture or learning algorithm.</p>
</blockquote>

<p><a href="https://arxiv.org/pdf/1312.5602v1.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/1312.5602v1.pdf</a></p>

<blockquote>
  <p>We could represent our Q-function with a neural network, that takes
  the state (four game screens) and action as input and outputs the
  corresponding Q-value. Alternatively we could take only game screens
  as input and output the Q-value for each possible action. This
  approach has the advantage, that if we want to perform a Q-value
  update or pick the action with highest Q-value, we only have to do one
  forward pass through the network and have all Q-values for all actions
  immediately available.</p>
</blockquote>

<p><a href="https://neuro.cs.ut.ee/demystifying-deep-reinforcement-learning/" rel="nofollow noreferrer">https://neuro.cs.ut.ee/demystifying-deep-reinforcement-learning/</a></p>
The huge amount of states in q-learning calculation
<p>I implemented a 3x3 OX game by q-learning ( it works perfectly in AI v.s AI and AI v.s Human), but I can't go one step further to 4x4 OX game since it will eat up all my PC memory and crash.</p>

<p>Here is my current problem:
<a href="https://stackoverflow.com/questions/56231392/access-violation-in-huge-array">Access violation in huge array?</a></p>

<p>In my understanding, a 3x3 OX game has a total 3(space, white, black) ^ 9 = 19683 possible states. ( same pattern different angle still count )</p>

<p>For a 4x4 OX game, the total state will be 3 ^ 16 = 43,046,721</p>

<p>For a regular go game, 15x15 board, the total state will be 3 ^ 225 ~ 2.5 x 10^107 </p>

<p>Q1. I want to know my calculation is correct or not. ( for 4x4 OX game, I need a 3^16 array ? )</p>

<p>Q2. Since I need to calculate each Q value ( for each state, each action), I need such a large number of array, is it expected? any way to avoid it?</p>