The proposed counter mechanism relies on being able to manually identify entities in the environment, such as the box in the push-box environment. This has limited applicability to real-world problems with large-dimensional or visual state spaces, in which entities are not obvious a priori. Being able to explicitly count the number of times an agent has experienced an entity in a specific configuration is not a realistic expectation for interesting, real-world problems. Therefore, it is unclear how this method can be applied beyond simple tabular settings and video games.
Similarly, it seems that deciding which subspaces are equivalent requires a significant amount of domain knowledge into each problem, and does not seem to be generally applicable.
Why not benchmark against QMIX + RND, since both are tested independently?
Other suggestions:
Typo on p. 3 "tuple is then store in a replay memory" -> stored
Why was the number of steps between updates for EITI and EDTI held constant at 64,000? How many steps between updates were used for the proposed technique?
3 "tuple is then store in a replay memory" -> stored Why was the number of steps between updates for EITI and EDTI held constant at 64,000? How many steps between updates were used for the proposed technique?