boost compared to CPU-ELPA1 using 24 CPU cores. The efficiency of CPU-ELPA2 on various distributed-memory CPU platforms [@elpa_marek_2014; @elpa_gutheil_2014; @elpa_cook_2018] indicates that GPU-ELPA2 may outperform GPU-ELPA1 on multiple nodes. Historically, the GPU porting of ELPA2 dates back to 2013, when Peter Messmer from NVIDIA programmed the first version of GPU-ELPA2. Then the code was refactored and merged into the mainline version of ELPA, and has been available in released versions of the ELPA eigensolver library since 2016. In this paper, we report our latest optimizations and developments that enable a performance improvement on distributed-memory, GPU-accelerated architectures. Specifically, several synchronizations and memory transfers between CPUs and GPUs have been optimized. Additionally, some kernels in one of the major computational steps, the tridiagonal-to-banded back-transformation of eigenvectors (Eq. \[eq:2stage\_bkwd1\]), have been rewritten. The rest of this paper is organized as follows. First, we briefly review the two-stage diagonalization algorithm, in particular the tridiagonal-to-banded back-transformation of eigenvectors and its CPU implementation in the ELPA library. Next, we outline the GPU acceleration strategies employed in GPU-ELPA2, and elaborate on our CUDA implementation of the tridiagonal-to-banded back-transformation of eigenvectors, which is essentially a GPU extension of the algorithm in Refs. [@elpa_auckenthaler_2011; @elpa_auckenthaler_2013]. Finally, we demonstrate the 