the phase factor $\exp(i\phi)$ can be introduced to $|\tilde{\epsilon}_a \rangle = |\epsilon_i\rangle\exp(i\phi_a)$ arbitrarily the values of the matrix elements $\langle  \tilde{\epsilon}_i | \lambda_j \rangle$ will vary with the phase of $|\epsilon_i\rangle$s. Now consider adding a perturbative electric field . The Hamiltonian then becomes $H = H_0 + E_x x$, and we have $UHU^\dagger = UH_0U^\dagger + E_x ( x + i U\partial_{k_x} U^\dagger)$ where, in this rotated frame, $H_0$ is diagonal, and we have an additional $i U\partial_{k_x}U^\dagger$. In order to figure out the lab frame spin$\otimes$pseudospin ‘direction’ where this contribution points to, we transform the $i U\partial_{k_x}U^\dagger$ piece *without the diagonal elements* back to the laboratory frame . The reason for the removal of the diagonal elements will become apparent later. With the diagonal elements in place, we have $U^\dagger (i U\partial_k U^\dagger) U = -i U^\dagger\partial_k U$. (We have dropped the suffix $x$ from $k_x$ and $E_x$ for notational simplicity)

We stress that $-i U^\dagger\partial_k U$ has the same numerical matrix elements in the laboratory frame regardless of the phases of the $\langle \lambda_i | \epsilon_j \rangle$. This is because

$$\begin{aligned}
    && -i U^\dagger \partial_k U  \\
    &=& -i |\lambda_a \rangle \langle \lambda_a|\epsilon_b \rangle \langle \partial_k \epsilon_b|\lambda_c \rangle 