Little is known about his early life, except that he was from Flanders, from either the vicinity of Ghent or Weert, near Antwerp. As a boy he went to Avellino in southern Italy, near Naples, where he became a choir boy in the chapel of Maria di Cardona, Marchesa of Padulla. Maria was the wife of Francesco d'Este, Marchese di Massalombarda, a captain under Charles V; Francesco was a son of the notorious Lucrezia Borgia, and her husband Alfonso I d'Este. Francesco was often in France and adjacent areas on military campaigns, and as an adjunct to these adventures he brought musically talented youths back to Italy with him. Wert's association with the Este family was to endure through most of his life.