things got so bad that nathaniel gable the second , fitzwilliam 's eldest son , moved out of the house and back to boston , although i suppose that could be due to the family 's interest in manufacturing over imports .