Wilbye was the son of a tanner. He was supported financially by a rich family called Kytson. The Kytsons were very musical and had lots of musical instruments and sheet music in their house. It is thought that he went with Elizabeth Cornwallis to Hengrave Hall near Bury St. Edmunds in around 1594 when she married Sir Thomas Kytson the Younger. When Elizabeth died, Wilbye moved to Colchester where he lived with Elizabeth’s youngest daughter with whom he had a long friendship. When Wilbye died he was quite rich.