not one of the four mackenzie brothers had managed to marry without some kind of scandal , and the two older brothers-one of whom was daniel 's father-had taken some of the most notorious courtesans in england as their lovers .