Title: A Massively Parallel Benchmark for Safe Dexterous Manipulation. Abstract: Safe Reinforcement Learning (Safe RL) aims to maximize expected total rewards and avoids violation of certain constraints at the same time. Many constrained environments have been designed to evaluate Safe RL algorithms, but they are more focused on simple navigation tasks and have tremendous gaps with the real world. Meanwhile, dexterous manipulation is a challenging topic in the field of robotics, and places high demands on safety constraints to ensure reliable manipulation in the real world. Consequently, we propose Safety DexterousHands, a massively parallel physical benchmark to facilitate experimental validation in Safe RL research. Safety DexterousHands is built in the Isaac Gym, a GPU-level parallel simulator that enables highly efficient RL training. We designed a series of challenging dexterous manipulation tasks around the safety constraints. To the best of our knowledge, Safety DexterousHands is the first large-scale benchmark focused on safe dexterous manipulation, offering 10+ manipulators and 100+ task combinations. Our experimental results show that Safe RL algorithms can perfectly solve the safe dexterous manipulation task by exploiting the sparse cost penalty, while unsafe RL algorithms struggle to solve most tasks without causing disruption. We expect that this benchmark can deliver a reliable and comprehensive evaluation for Safe RL algorithms and promote a integration of Safe RL and dexterous manipulation.
