William Barclay Foster had serious financial troubles. In 1827, The White Cottage and other Foster properties were seized by the government when Foster failed to pay the mortgages. Stephen thereafter was moved about, sometimes living with relatives. His mother remembered him later in her life as a boy marching about with a feather in his cap and pounding on a drum while whistling "Auld Lang Syne". She noted that there was something "original" about the boy.