<p>You can find the answer in research paper about those algorithms, because when a new algorithm been proposed we usually need the experiments to show the evident that it have advantage over other algorithm.</p>
<p>The most commonly used evaluation method in research paper about RL algorithms is <strong>average return</strong> (note not reward, return is accumulated reward, is like the score in game) over timesteps, and there many way you can average the return, e.g average wrt different hyperparameters like in <a href="https://arxiv.org/abs/1801.01290" rel="nofollow noreferrer">Soft Actor-Critic paper</a>'s comparative evaluation average wrt different random seeds (initialize the model):</p>
<blockquote>
<p>Figure 1 shows the total average return of evaluation rolloutsduring
training for DDPG, PPO, and TD3.  We train fivedifferent instances of
each algorithm with different randomseeds, with each performing one
evaluation rollout every1000 environment steps. The solid curves
corresponds to themean and the shaded region to the minimum and
maximumreturns over the five trials.</p>
</blockquote>
<p><a href="https://i.stack.imgur.com/xbocp.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/xbocp.png" alt="enter image description here" /></a></p>
<p>And we usually want compare the performance of many algorithms not only on one task but diverse set of tasks (i.e Benchmark), because algorithms may have some form of inductive bias for them to better at some form of tasks but worse on other tasks, e.g in <a href="https://arxiv.org/abs/2009.04416" rel="nofollow noreferrer">Phasic Policy Gradient paper</a>'s experiments comparison to PPO:</p>
<blockquote>
<p>We  report  results  on  the  environments  in  Procgen  Benchmark
(Cobbe  et  al.,2019).  This benchmark was designed to be highly
diverse, and we expect improvements on this benchmark to transfer well
to many other RL environment</p>
</blockquote>
<p><a href="https://i.stack.imgur.com/lwKFF.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/lwKFF.png" alt="enter image description here" /></a></p>
Understanding and Evaluating different methods in Reinforcement Learning
<p>I have been trying to implement the Reinforcement learning algorithm on Python using different variants like <code>Q-learning</code>, <code>Deep Q-Network</code>, <code>Double DQN</code> and <code>Dueling Double DQN</code>. Consider a cart-pole example and to evaluate the performance of each of these variants, I can think of plotting <code>sum of rewards</code> to <code>number of episodes</code> <a href="https://i.stack.imgur.com/kpV8I.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/kpV8I.png" alt="evaluating a Reinforcement learning model" /></a> (attaching a picture of the plot) and the actual graphical output where how well the pole is stable while the cart is moving.</p>
<p>But these two evaluations are not really of interest in terms to explain the better variants quantitatively. I am new to the Reinforcement learning and trying to understand if any other ways to compare different variants of RL models on the same problem.</p>
<p>I am referring to the colab link <a href="https://colab.research.google.com/github/ageron/handson-ml2/blob/master/18_reinforcement_learning.ipynb#scrollTo=MR0z7tfo3k9C" rel="nofollow noreferrer">https://colab.research.google.com/github/ageron/handson-ml2/blob/master/18_reinforcement_learning.ipynb#scrollTo=MR0z7tfo3k9C</a> for the code on all the variants of cart pole example.</p>