Title: Learning Control Lyapunov Functions For High-dimensional Unknown Systems using Guided Iterative State Space Exploration. Abstract: Designing stable controllers in complex, high-dimensional systems with unknown dynamics is a critical problem when we deploy robots in the real world. Prior works use learning-based control Lyapunov functions (CLFs) or adaptive control to derive such controllers, but they suffer from two significant challenges: scalability and model transparency. This paper proposes a general framework to jointly learn the local dynamics, a stable controller, and the corresponding CLF in high-dimensional unknown systems. Our approach, GIE-CLF, does not need any knowledge of the environment, such as the dynamics, reward functions, etc, and can scale up to high dimensional systems using only local knowledge of the dynamics inside a trusted tunnel instead of global knowledge required by other methods. We provide theoretical guarantees for our framework and demonstrate it on highly complex systems including a high-fidelity F-16 jet aircraft model that has a 16-dimensional state space and a 4-dimensional input space. Experimental results show that GIE-CLF significantly outperforms prior works in reinforcement learning and imitation learning. We also show that our algorithm can also be extended to learn other control certificate functions for unknown systems.