I think Ann Landers had the last word on this - or was it her sister Abby?  
So you're having an affair with a married man and you think he's going to divorce his wife and marry you.  Let's suppose that actually happens (almost never), who do you end up being married to?  A man who cheats on his wife!