these houses were owned by lawyers and doctors from washington and baltimore who for the most part kept expensive sailing yachts in the river sunday harbor and used the houses as second homes and investments .