What he did was not the crime of the century as we are all (women and men) potentially vulnerable to our feelings, especially when we are weakened by alcohol. However, he has been untrue to his spouse (their problem, and you simply don't know that full background to, and state of, their relationship) AND the two couples have to end their contact.