Baldwin's matrimonial ventures periodically created sensations. He was married four times, the first two marriages ending in divorce. His third wife, Jennie Dexter, was a 16-year-old child at the time of marriage. She died of tuberculosis at the age of 23, leaving him with a five-year-old daughter. Baldwin remarried another 16-year old dark-haired, dark-eyed child called Lillie Bennett who had a striking resemblance to Jennie. The Queen Anne Cottage was built as a honeymoon present by Lillie's father. They separated after less than two years but never divorced. He reportedly had many affairs with young women in his older years. He was sued by four women for breach of promise of marriage. His stature as a celebrity was such that at age 56, when he married Bennett in San Francisco, the wedding drew coast-to-coast press coverage. In the same year, he was sued by a 16-year-old girl who was awarded $75,000 in damages.