<p>Yes, it is possible to use OpenAI gym environments for multi-agent games. Although in the OpenAI gym community <a href="https://github.com/openai/gym/issues/934" rel="noreferrer">there is no standardized interface</a> for multi-agent environments, it is easy enough to build an OpenAI gym that supports this. For instance, in OpenAI's <a href="https://arxiv.org/pdf/1706.02275.pdf" rel="noreferrer">recent work</a> on multi-agent particle environments <a href="https://github.com/openai/multiagent-particle-envs/blob/master/multiagent/environment.py" rel="noreferrer">they make a multi-agent environment</a> that inherits from <code>gym.Env</code> which takes the following form:</p>

<pre class="lang-py prettyprint-override"><code>class MultiAgentEnv(gym.Env):

    def step(self, action_n):
        obs_n    = list()
        reward_n = list()
        done_n   = list()
        info_n   = {'n': []}
        # ...
        return obs_n, reward_n, done_n, info_n
</code></pre>

<p>We can see that the <code>step</code> function takes a list of actions (one for each agent) and returns a list of observations, list of rewards, list of dones, while stepping the environment forwards. This interface is representative of <a href="https://www2.cs.duke.edu/courses/spring07/cps296.3/littman94markov.pdf" rel="noreferrer">Markov Game</a>, in which all agents take actions <em>at the same time</em> and each observe their own subsequent observation, reward.</p>

<p>However, this kind of Markov Game interface may not be suitable for all multi-agent environments. In particular, turn-based games (such as card games) might be better cast as an <em>alternating</em> Markov Game, in which agents <em>take turns</em> (i.e. actions) one at a time. For this kind of environment, you may need to include which agent's turn it is in the representation of state, and your step function would then just take a single action, and return a single observation, reward and done.</p>
Openai gym environment for multi-agent games
<p>Is it possible to use <a href="https://openai.com/" rel="noreferrer">openai</a>'s <a href="https://gym.openai.com/docs" rel="noreferrer">gym environments</a> for multi-agent games? Specifically, I would like to model a card game with four players (agents). The player scoring a turn starts the next turn. How would I model the necessary coordination between the players (e.g. who's turn it is next)? Ultimately, I would like to use reinforcement learning on four agents that play against each other.</p>