<p>PPO is a simple algorithm, which falls into policy optimization algorithms class (as opposed to value-based methods such as DQN). If you "know" RL basics (I mean if you have at least read thoughtfully some first chapters of <a href="http://www.incompleteideas.net/book/the-book-2nd.html" rel="noreferrer">Sutton's book</a> for example), then a first logical step is to get familiar with policy gradient algorithms. You can read <a href="https://papers.nips.cc/paper/1713-policy-gradient-methods-for-reinforcement-learning-with-function-approximation.pdf" rel="noreferrer">this paper</a> or chapter 13 of <a href="http://www.incompleteideas.net/book/the-book-2nd.html" rel="noreferrer">Sutton's book</a> new edition. Additionally, you may also read <a href="https://arxiv.org/pdf/1502.05477.pdf" rel="noreferrer">this paper</a> on TRPO, which is a previous work from PPO's first author (this paper has numerous notational mistakes; just note). Hope that helps.  --Mehdi</p>
What is the way to understand Proximal Policy Optimization Algorithm in RL?
<p>I know the basics of Reinforcement Learning, but what terms it's necessary to understand to be able read <a href="https://arxiv.org/abs/1707.06347" rel="noreferrer">arxiv PPO paper</a> ?</p>

<p>What is the roadmap to learn and use <a href="https://blog.openai.com/openai-baselines-ppo/" rel="noreferrer">PPO</a> ?</p>