Accurate, robust and drift-free global pose estimation is a fundamental problem for autonomous vehicles. In this work, we propose a global drift-free map-based localization method for estimating the global poses of autonomous vehicles that integrates visual–inertial odometry and global localization with respect to a pre-built map. In contrast to previous work on visual–inertial localization, the global pre-built map provides global information to eliminate drift and assists in obtaining the global pose. Additionally, in order to ensure the local odometry frame and the global map frame can be aligned accurately, we augment the transformation between these two frames into the state vector and use a global pose-graph optimization for online estimation. Extensive evaluations on public datasets and real-world experiments demonstrate the effectiveness of the proposed method. The proposed method can provide accurate global pose-estimation results in different scenarios. The experimental results are compared against the mainstream map-based localization method, revealing that the proposed approach is more accurate and consistent than other methods.