<p>There are two closely related works in recent two years:</p>
<p>[1] Boutilier, Craig, et al. &quot;Planning and learning with stochastic action sets.&quot; arXiv preprint arXiv:1805.02363 (2018).</p>
<p>[2] Chandak, Yash, et al. &quot;Reinforcement Learning When All Actions Are Not Always Available.&quot; AAAI. 2020.</p>
State dependent action set in reinforcement learning
<p>How do people deal with problems where the legal actions in different states are different? In my case I have about 10 actions total, the legal actions are not overlapping, meaning that in certain states, the same 3 states are always legal, and those states are never legal in other types of states. </p>

<p>I'm also interested in see if the solutions would be different if the legal actions were overlapping. </p>

<p>For Q learning (where my network gives me the values for state/action pairs), I was thinking maybe I could just be careful about which Q value to choose when I'm constructing the target value. (ie instead of choosing the max, I choose the max among legal actions...)</p>

<p>For Policy-Gradient type of methods I'm less sure of what the appropriate setup is. Is it okay to just mask the output layer when computing the loss? </p>