Work began on Kirby Muxloe Castle in October 1480, under the direction of Roger Bowlott, who was Lord Hasting's steward and became the clerk of works for the project. Most of the existing manor house was protected during the work, and probably continued in use while the new castle was built around it. A workshop able to make up to 100,000 bricks a week was constructed by Antony Yzebrond from Flanders. A brook was redirected to feed the new moat and a garden was established alongside the castle. Work intensified in early 1481, with the stonework being supervised by John Cowper, a master mason. The work progressed at speed and, at peak periods, over 40 men were working on the site. It is uncertain exactly how much of the castle was constructed, although at least one corner tower had been nearly completed by 1483 and the ground floor of the gatehouse laid down.