weakness of the paper is that experiments are limited to a single task. That said, they compare against two reasonable baselines (CPO, and including the constraint as a negative reward). While the formal definition of the constrained objective in L149 - L155 is appreciated, it might be made a bit more clear by avoiding some notation. For example, instead of defining a new function I(x,y) (L151), you could simply use \delta(x >= y), stating that \delta is the Dirac delta function. A visualization of Eq 1 might be helpful to provide intuition. Minutia * Don't start sentences with citation. For example, (L79): "[32] proposed ..." * Stylistically, it's a bit odd to use a *lower* bound on the *cost* as a constraint. Usually, "costs" are minimized, so we'd want an upper bound. * Second part of Def 4.2 (L148) is redundant. If Pr(X >= \gamma) <= \rho, then it's vacuously true that Pr(X <= \gamma) >= 1 - \rho. Also, one of these two terms should be a strict inequality. * Include the definition of S (L154.5) on its own line (or use \mbox). * Label the X and Y axes of plots. This paper makes an important step towards safe RL. While this paper builds upon much previous work, it clearly documents and discusses comparisons to previous work. While the results are principally theoretical, I believe it will inspire both more theoretical work and practical applications.
* Label the X and Y axes of plots. This paper makes an important step towards safe RL. While this paper builds upon much previous work, it clearly documents and discusses comparisons to previous work. While the results are principally theoretical, I believe it will inspire both more theoretical work and practical applications.