Title: Decentralized Robust V-learning for Solving Markov Games with Model Uncertainty. Abstract: Markov game is a popular reinforcement learning framework for modeling competitive players in a dynamic environment. However, most of the existing works on Markov game focus on computing a certain equilibrium following uncertain interactions among the players, but ignores the uncertainty of the environment model, which is ubiquitous in practical scenarios.  In this work, we develop a tractable solution to Markov games with model uncertainty. Specifically, we propose a new and tractable notion of robust correlated equilibrium for Markov games with environment model uncertainty. In particular, we prove that robust correlated equilibrium has a simple modification structure, and its characterization of equilibrium critically depends on the environment model uncertainty. Moreover, we propose the first fully-decentralized sample-based algorithm for computing such robust correlated equilibrium. Our analysis proves that the algorithm achieves the polynomial sample complexity $\widetilde{\mathcal{O}}( SA^2 H^5 p_{\min}^{-2}\epsilon^{-2})$ for computing an approximate robust correlated equilibrium with $\epsilon$ accuracy. 