Ignoratio elenchi
A red herring, the unrelated alleged affair, attempts to change the subject away from the popular policies.  However, if the original discussion were of the prime minister's public integrity (encompassing both popularity and conduct), this argument could be perfectly valid. Such as if a politician's election campaign revolved around family values. Then an argument about such an affair would be valid, as it is in direct conflict with a family values platform.