The then 29-year-old Williams became friendly with 32-year-old Agnes Powell, who worked as a secretary with the Temperance Society. She was the eldest of a family of three sons and four daughters of Captain Francis Powell of Kent, who was prominent in local Masonic and Conservative political circles. Williams and Agnes Powell married in 1898 in the face of the strongest opposition of her father, who refused to give his consent and thereafter refused to receive Williams. They had five children; the first, Henry Francis Sylvestre, was born the following year.