and eigenvectors of randomly generated real and complex matrices of size $N$ = 40,000 to 100,000 are computed with up to 64 nodes. As already demonstrated in published benchmarks [@elpa_marek_2014; @elpa_gutheil_2014; @elpa_cook_2018; @elpa_auckenthaler_2013], CPU-ELPA2 greatly outperforms CPU-ELPA1 in terms of performance and scalability. The performance difference between GPU-ELPA1 and GPU-ELPA2 is rather small. For small node counts, GPU-ELPA1 is marginally faster than GPU-ELPA2, which agrees with the published single-node tests [@elpa_kus_2019b]. What has not been tested previously is the performance on multiple nodes. It turns out that GPU-ELPA2 becomes faster than GPU-ELPA1 as the node count increases. The crossover point depends on the dimension of the problem, e.g., 4 nodes in the $N$ = 40,000 real case, and 16 nodes in the $N$ = 100,000 complex case. When using 64 nodes, the speedup of GPU-ELPA2 over GPU-ELPA1, averaged over all matrix sizes in Fig. \[fig:all\_talos\], is 2.2x.

In Fig. \[fig:all\_talos\], the speedup enabled by the GPUs ranges from no speedup at all to 3.3x. Three general trends emerge: (1) For the same matrix size, the speedup becomes smaller as the node count increases. For small $N$, CPU-ELPA2 can even be faster than the GPU-accelerated solvers, thanks to the near-optimal strong 